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I.  IN TEODOCTION 


This  thesis  concentrates  on  the  Marine  Corps  Development 
and  Education  Command,  Research  and  Development  Program 
Prioritiration  Process.  The  process  was  originally  proposed 
to  assist  in  developing  the  Marine  Corps'  input  to  the 
Navy's  Research,  Development,  Test  and  Evaluation  (F  DT  SZ , N) 
Program  Chjective  Memorandum  (POM) .  The  process  combines 
subjective  evaluations  of  the  Marine  Corps'  identified 
mission  deficiencies  and  proposed  RDTSE  programs,  and 
produces  a  RDTSE  program  rank  ordering  according  to  mission 
deficiency  accomp lishment  importance.  The  Department  of 
Defense  acguisition  system  is  reviewed  as  it  establishes  the 
base  for  Marine  Corps  RDTSE  program  acguisition.  Review  of 
current  acguisition  procedures  reveal  that  analytic  models 
are  used  in  support  cf  the  Procurement  Marine  Corps  (PMC) 
POM  tut  net  for  Marine  Corps  RDTSE,  N  ?0M  input. 

The  literature  concerning  research  and  development 
program  selection  models  is  reviewed.  The  review  shows  that 
a  wide  range  of  program  selection  models  are  proposed  and 
accepted  by  industry.  The  literature  lends  credence  tc  the 
MCDEC  process  by  allowing  process  classification  with  the 
majority  cf  industry  accepted  models.  Mathematical  program¬ 
ming  enhancement  to  this  classification  is  suggested  by  the 
literature  and  is  illustrated  in  this  investigation.  Linear 
programming  formulations  are  used  to  duplicate  the  MCDEC 
process  and  suggest  mathematical  programming  advantages 
under  tudget  reduction  conditions.  A  MCDEC  produced  process 
test  data  set  is  used  to  further  illustrate  the  linear 
programming  formulations.  A  second  linear  programming 
approach  is  suggested  which  concentrates  on  the  number  of 
mission  deficiencies  accomplished  rather  than  the  FDTSZ 


programs  and  presents  a  favorable  program  selection  model 
alternative  f - r  the  sample  data  set. 

Further  investigation  is  required  to  establish  ether 
possible  alternatives  suggested  by  the  literature  ana  for 
the  implementation  of  a  Marine  Corps  HDT5E  program  selection 
process.  Further  consideration  of  the  costs  associated  with 
software,  data  collection,  and  manpower  requirements  will  be 
necessar}  as  well  as  comparing  these  costs  with  the  benefits 
received  from  a  process  implementation. 
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II.  eackgegond 

A.  I KTECDUCTION 

The  Flaming/  Programming/  and  Budgeting  System  (FPES) 
structures  the  defense  systems  acquisition  procedure  within 
the  Department  of  Defense  (DoD).  This  structure  is  a  step¬ 
wise  review  and  decision  process  conducted  at  various  levels 
in  the  EoD  hierarchy.  The  process  translates  validated 
defense  mission  needs  into  budget  allocations  which  fund 
programs  that  are  expected  to  overcome  the  stated  needs.  In 
this  thesis  the  hierarchical  level  of  concern  descends  from 
the  Office  of  the  Secretary  of  Defense  (OSD),  through  the 
Department  of  the  Navy  (DoN),  to  the  Marine  Corps  and  the 
Navy.  The  analysis  concentrates  on  the  initial  PPBS  steps 
taken  at  this  level,  and  specifically  the  Research, 
Development,  Test,  and  Evaluation  (F.DT&E)  acquisition 
program  pricriti zaticr  and  funding  profile  process  used  by 
the  Marine  Corps. 

E.  SYSTEMS  ACQDISIT ICN  IN  DSMC 
1  .  Fes ronsibili ties 

Specific  respcnsib ilities  are  assigned  the  Marine 
Corps  by  law.i  Figure  2.1  depicts  three  levels  uf  ECTSE  and 
systems  acquisition  management  that  assist  in  accomplishing 
these  responsibilities.  The  RDTSE  and  system  acquisition 
responsibilities  stated  in  MCO  5000. 10A  [Ref.  1]  first 


‘The  National  Security  Act  of  1947,  as  amended,  and 
subsequent  DoD  and  SecNav  directives  assign  the  Marine  Corps 
general  and  specific  responsibilities  for  developing  the 
tactics,  techniques,  and  equipment  used  by  amphibious 
landing  forces.  The  Marine  Corps  is  directed  to  coordinate 
and  request  appropriate  assistance  for  this  development  with 
the  Navy  and  other  military  services. 


designates  the  Assistant  Commandant  of  the  Marine  Corps 
(ACMC )  as  the  acquisition  executive  and  then  assigns  him  the 
responsibility  for  overall  integration  and  unification  of 
the  management  process.  The  Deputy  Chief  of  Staff  ( CC/S) 
for  Research,  Development,  and  Studies  (RDSS)  assists  the 
ACMC  ty  acting  as  the  coordinator  of  staff  activities 
involved  in  RDTSE  and  system  acquisition  from  program  initi¬ 
ation  to  approval  for  service  use.  The  Commanding  General, 
Marine  Corps  Development  and  Education  Command  {CG,  MCDEC) 
is  the  field  representative  of  the  Commandant  of  the  Marine 


Corps  (CMC) 

in  RDT 

SE  efforts  and  is 

responsible 

for  the 

field  exec 

ution  and 

coordination  of 

Marine  Corps* 

RDTSE 

activities 

to  supp 

ert  the  systems 

acquisit ion. 

This 

includes  coordinating  Marine  Corps  support  needs  with  ether 
military  services,  as  well  as  reporting  on  RDTSE  efforts  of 
ether  services,  agencies,  and  governments. 


Figure  2.  1  RDTSE  and  System  Acquisition  levels. 


The  deputy  chiefs  of  staff  or  directors  of  major 
staff  offices  at  Headguarters  Marine  Corps  (HQMC)  are 
assigned  as  program  sponsors  for  specific  Marine  Corps 
mission  areas  and  are  intimately  involved  in  the  acquisition 


process.  The  program  sponsors  are  responsible  for  the 
continuous  analysis  of  their  mission  areas  and  the  overall 
planning,  coordination,  and  direction  of  related  program 
acquisitions.  These  responsibilities  are  accomplished  in 
coordination  with  DC/S  RDES  and  CG,  MCDEC. 

2.  Analysis  of  Needs 

The  directed  continual  analysis  of  mission  areas  by 
the  program  sponsors,  kith  input  from  the  Fleet  Marine  Force 
(FMF)  ,  initially  identifies  operational  mission  needs. 
These  needs,  or  equivalently  mission  deficiencies,  are 
formally  refined,  additional  needs  identified,  and  related 
new  concepts  formulated  by  Mission  Area  Analysis  ( MA  A)  . 
MAAs  are  assessments  of  current  or  projected  Marine  Cores 
capabilities  within  the  specific  mission  areas  and  elements 
listed  in  Table  I  and  are  conducted  by  H2'1C  or  MCE  EC  staff 
sections  in  coordination  with  the  program  sponsors  and  C3, 
MCDEC .  The  MAA  details  the  mission  elements  of  "able  I  into 
general  operational  missions  and  then  further  into  specific 
operational  missions.  These  specific  missions  can  then  be 
matched  against  specific  resources  for  their  accomplishment. 
If  nc  resources  are  available  then  a  deficiency  is  identi¬ 
fied  and  recommendations  for  overcoming  the  deficiency  are 
presented.  SECNAVINST  5000. 1A  [Ref.  2]  and  "A  Guide  for  the 
Eerfcrmance  of  USMC  Mission  Area  Analysis"  [Ref.  3]  detail 
the  requirements  and  methodology  of  conducting  MAAs.  Each 
identified  deficiency  is  formulated  into  a  Justification  for 
Major  System  New  Start  (JMSNS)  or  a  Required  Operational 
Capability  (ROC)  according  to  the  Acquisition  Category 
(ACAT)  .  This  documentation  then  forms  the  basis  for  initi¬ 
ating  RDTSE  funding  and  program  acquisition. 


TABLE  I 

Marine  Corps  Mission  Areas  and  Elements 


UJ 

hrt 

ELEMENT 

TITLE 

2  10 

211 

212 

213 

214 

215 

216 

Land  Warfare 

Close  Combat 

Fire  Support 

Ground  Air  Defense 

Mine  Warfare 

Land  Combat  Support 

Land  Combat  Service  Support 

220 

221 

222 

224 

225 

Air  Warfare 

Counter  Air 

Offensive  Air 

Defense  Suppression 

Air  Warfare  Support 

230 

235 

Naval  Warfare 

Amphibious  Warfare 

240 

243 

Tactical  Nuclear  Warfare 

Defensive  Tactical  N-Warfare 

2  50 

252 

255 

256 

257 

Theater  and  Tactical  C3I 

Tactical  Command  Control 
and  Intelligence  Systems 
Surveillance,  Reconnaissance  £ 
Targets  for  Acquisition 

Tactical  Communications 

Electronic  Warfare  £  Counter  C3I 

260 

261 

262 

264 

Mobi lity 

Airlift 

Seal  if t 

Refu  eling 

320 

321 

322 

324 

Defense  Wide  C3I  Support 

Navigation  £  Position  Fixing 
Support  and  Base  Communication 

COM 5  EC 

430 

43  1 

Non- System  Training  Devices 
Training  Devices/Simulators 

450 

453 

454 

Test  and  Evaluation  Support 

Joint  Test  and  Evaluation 

Other  Test  and  Evaluation 

460 

46  1 

International  Cooperative  RDTSE 
Standardization  £  Interoperability 

4  70 

47  1 

Management  Support 

General  Management  Support 

520 

521 

522 

523 

Exploratory  Development 

Electronic  £  Physical  Sciences 
Environmental  £  Life  Sciences 
Engineering  Technology 

c 


BEVIES  2ND  DECISION  PROCESS 


1 .  tileston  es 

Four  formal  decision  points  are  established  within 
the  program  acquisition  process  and  are  designated  mile¬ 
stones  0 ,  I,  II,  and  III.  Positive  approval  is  necessary  at 
each  milestone  for  passage  of  a  program  from  conception  to 
operational  deployment.  Milestone  0  signifies  program  initi¬ 
ation  cn  approval  of  a  JMSNS  cr  a  ROC.  A  favorable  mile¬ 
stone  I  decision  gives  the  approval  to  demonstrate  selected 
alternatives.  Then  a  milestone  II  approval  authorizes  full 
scale  development  ard  limited  production  for  operational 
testing  and  evaluation.  Finally,  milestone  III  passage  is 
necessary  fcr  production  release  and  approval  for  service 
use.  Marine  Corps'  input  to  the  R D T £ S , N  Program  Objective 
Memorandum  (POM)  is  reguired  throughout  this  process  until 
milestone  III,  after  which  acquisition  funding  comes  under 
the  Procurement  Marine  Corps  (PMC)  POM. 

2 .  Phases 

The  review  steps  of  the  acquisition  process,  shewn 
in  Figure  2.2  ,  are  described  as  phases  separated  by  the 
decision  milestones.  The  initial  reviews  and  analysis 
previously  discussed  are  included  in  the  first,  or  research 
phase.  Studies  and  MAAs  conducted  in  this  phase  identify 
mission  deficiencies  that,  in  turn,  generate  JMSNSs  cr  FOCs. 
These  documents  assess  the  projected  threat,  state  the 
mission  element  deficiency,  identify  the  existing  DCD  capa¬ 
bilities,  assess  the  impact  of  not  acquiring  or  maintaining 
the  capabilities,  and  finally  provide  a  program  plar  to 
identify  and  explore  competitive  alternatives. 

Approval  of  the  JMSNS  or  ROC  at  milestone  0  moves 
the  process  from  the  research  phase  to  the  program  initia¬ 
tion  phase.  The  objectives  of  this  phase  include  further 
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Phase 


Objective 


Fundin  g 


Figure  2.2  Marine  Corps  Acquisition  Process  Flow  Chart 


program  study  and  the  development  of  a  technological  base  to 
support  the  program  through  the  remaining  reviews  and  deci¬ 
sions.  Tasks  necessary  to  accomplish  these  objectives  are 
program  cost  analysis,  operational  effectiveness  studies, 
alternative  identification,  technical  and  economical  feasi¬ 
bility  studies,  risk  determination,  and  further  concept  and 
threat  analysis.  The  primary  documents  produced  during  this 
phase  are  the  System  Concept  Paper  {SC?)  for  major  pregrams 
or  the  Decision  Coordinating  Paper  (DCP)  for  less-than- ma jor 
programs.  These  documents  summarize  the  program  development 
to  date  and  address  mission  element  need,  technology  assess¬ 
ment,  program  description,  management  plan,  and  acquisition 
logistics  and  manpower. 

Approval  of  the  SCP  or  DC?  at  milestone  I  starts  the 
demonstration  and  validation  phase.  Viable  alternative 
systems  and  critical  subsystems  are  subjected  to  competitive 
demonstrations  during  this  phase.  Tasks  accomplished  in 
conjucticn  with  this  phase  include  reviews  of  acquisition 
strategies,  logistics,  manpower  and  training  planning,  as 
well  as  the  preparation  of  the  test  and  evaluation  master 
plan  (TEMP).  Major  consideration  is  given  to  the  thorough 
understanding  of  the  operational  need  and  the  evaluation  of 
all  alternatives.  Each  alternative’s  unit  cost  goal,  life 
cycle  cost,  technical  feasibility,  and  economic  realism  is 
reviewed.  When  the  demonstration  and  validation  phase 
produces  sufficient  evidence  that  the  preferred  system  can 
fulfill  all  necessary  capabilities,  and  that  technology 
exists  tc  produce  the  system,  the  program  sponsor  recommends 
approval  at  milestone  II. 

A  successfully  passed  milestone  II  initiates  the 
full  scale  development  (FSD )  phase.  This  phase  ensures  that 
the  engineering  and  production  design,  test  and  evaluation, 
personnel  and  training,  and  the  integrated  logistic  support 
planning  are  completed  prior  tc  moving  into  production.  Upon 


completion  of  all  research  ard  development  tn rough  this 
phase,  the  approval  for  service  use  is  given  at  milestone 
III.  This  decision  to  proceed  starts  the  production  and 
deployment  phase  and  also  designates  the  shifts  of  funding 
from  EDT6E,N  to  PMC  as  well  as  the  coordination  responsi¬ 
bility  from  the  DC/S  ED &S  to  the  Deputy  Chief  of  Staff  for 
Installations  and  Logistics. 

3 .  Categories 

All  mission-essential  acquisition  programs  in  the 
Marine  Corps  are  classified  as  either  a  major  or  less-than- 
major  program.  These  two  categories  imply  the  level  at 
which  a  milestone  decision  is  made.  Major  programs  are 
designated  according  to  their  funding  level  and  ether 
criteria,  and  may  he  assigned  by  the  Secretary  of  Defense 
(SecDef)  ,  Secretary  of  the  Navy  (SecNav) ,  or  the  Commandant 
of  the  Marine  Corps  (CMC).  Department  of  Defense 
Instruction  5000  .2  [Bef.  4],  SECNAVINST  5000.  1A  [Eef.  2], 
and  MCO  5000. 1  0A  [Eef.  1]  list  major  program  criteria  used 
in  these  designations.  Less- than- major  programs  are  simply 
all  ether  mission-essential  acquisition  programs  not  desig¬ 
nated  as  major  programs. 

4  .  Be  view  Councils 

The  level  of  hierarchy  that  designates  a  program  as 
major  also  becomes  the  final  decision  authority  for  that 
program  at  milestones  I,  II,  and  III.  The  decision  makers 
are  supported  by  recommendations  developed  by  the  appro¬ 
priate  review  councils  which  convene  for  these  three  mile¬ 
stones.  The  Defense  System  Acquisition  RevLew  Council 
(DSABC)  and  the  Department  of  the  Navy  System  Acquisition 
Review  Council  (DNSAEC)  provide  program  recommendations  to 
the  SecDef  and  the  SecNav,  respectively.  Similarly,  the 
Marine  Corps  System  Acquisition  Review  Council  (MSARC) 
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supper ts  the  CMC  or  all  aajor  programs  including  these 
subject  to  higher  review.  The  MSA SC  consists  of  a  board  of 
general  officers,  which  is  chaired  by  the  ACMC  and  includes 
the  CG,  MCEEC,  DC/S  REES,  and  other  aajor  HQMC  staff  chiefs 
and  directors.  This  council  receives  a  comprehensive  review 
cf  the  research  and  development  concerning  the  acquisition 
program  and  formulates  recommendations  for  submission  to  the 
CMC. 

The  less-than-majo r  programs  undergo  a  similar 
review  process  by  an  In  Process  Review  (IPS)  Committee.  This 
committee  is  chaired  by  the  DC/S  HDDS  and  has  representation 
from  all  aajor  HQMC  staff  and  the  CG,  MCDEC.  The  ACKC  is  the 
decision  authority  for  less-than-ma jor  programs,  and  the  IPR 
committee  submits  recommendations  to  the  ACMC  for  the  acqui¬ 
sition  pregram  decision. 

E.  0 SMC  RDTSE  PROGRAMMING  AND  BUDGETING 

All  research,  development,  testing,  and  analysis 
described  in  this  chapter  depend  critically  on 
Congressionally-approved  allocations.  The  estimation  and 
submission  cf  funding  requests  are  coordinated  by  the  DC/S 
EDSS  for  each  fiscal  year,  and  this  process  involves  all 
Chiefs  cf  Staff  and  Directors  at  HQMC.  Initially  each 
program  sponsor  must  submit  a  prioritized  listing  and  accu¬ 
rate  funding  estimate  of  his  respective  acquisition 
programs.  The  programs  are  then  combined  and  subjectively 
prioritized  and  evaluated  through  a  sequence  of  committees. 
The  program  sponsors  present  each  proposal  before  a  Program 
Evaluation  Group  (PEG)  and  then  a  POM  Working  Group  (PWG)  . 
The  groups'  results  are  then  reviewed  and  evaluated  prior  to 
submission  to  the  CMC  for  approval.  The  approved  acquisition 
program  funding  profiles  levels  are  included  in  the  Navy 
portion  cf  the  Five  Tear  Defense  Plan  (FYDP)  and  become  the 


Marine  Corps'  input  tc  the  RDTSE, N  POM.  After  DoN,  DCD,  and 
Congressional  negotiations,  Marine  Corps  R DTE E  funds  are 
appropriated  by  Congress  as  part  of  the  RDTSE, N  budget. 

A  similar  cyclic  process  is  involved  in  the  development 
and  submission  of  the  PMC  PCM  for  the  allocation  of  funds 
for  the  production  and  deployment  phase  after  a  positive 
milestone  III  decision.  Cne  difference  between  the  two 
processes  is  the  existence  of  an  analytic  support  model  used 
in  the  PMC  process  and  the  current  absence  of  a  support 
model  for  the  RDTSE  process.  The  model  used  during  the  PMC 
POM  process  was  developed  under  civilian  contract  and  is 
described  in  the  final  contract  report,  "Decision-Analy tic 
Support  of  the  USMC  Program  Development:  A  Guide  tc  the 
bet  he  dolcgy "  £Ref.  5'.  Independent  of  the  PMC  model,  the 
CG,MCEEC  developed  a  process  with  the  objective  of  combining 
many  subjective  judgements  from  headquarters  and  operational 
levels  tc  prioritize  RDTSE  aeguisitioa  programs  and  provide 
suggestions  in  program  funding  profiles,  however,  this 
process  has  not  teen  implemented. 

E.  TEE  FEOELEM 

Defense  system  research  and  development  programs  indi¬ 
vidually  do  not  possess  an  obvious  numerical  quantity  that 
allows  mathematically  straightforward  program  comparisons  or 
funding  profile  optimization.  Inherent  then  tc  RDTSE 
program  and  funding  decisions  are  the  personal  judgments  of 
the  decision  maker.  These  subjective  decisions  rely  on  the 
information  and  analytic  support  available  to  the  individual 
and  supporting  staff  croup  prior  to  the  decision.  Marine 
Corps  RDTSE  acquisition  programs  are  prioritized  in  upper 
hierarchical  level  committees  and  currently  do  not  utilize 
mathematical  analytical  decision  support.  The  increasing 
fiscal  commitment  associated  with  each  new  defense  program 
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and  the  intense  competition  for  scarce  resources  imply  the 
critical  importance  cf  prioritization  decisions.  Tc  ensure 
presentation  of  the  lest  programs  for  funding  allocations  or 
to  optimize  the  portfolio  of  programs  presented  will  require 
assistance  from  analytical  decision  tools.  The  prioritiza¬ 
tion  process  proposed  by  MC  DEC  utilizes  a  model  composed  of 
linear  combinations  of  normalized  subjective  weights.  ether 
methods  cf  multi -at tribute  analysis  and  models  are  suggested 
in  the  literature  from  industrial  applications  and  theoret¬ 
ical  techniques.  Investigation  into  the  F.DTSE  program 
prioritization  process  is  warranted  and  may  provide  opti¬ 
mizing  techniques  which  will  assist  the  Marine  Corps  in 
EDTSE  resource  allocation  and  program  selection. 
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III.  IITZRAIOEE  review 


A.  IBTRCDOCTION 

The  importance  cf  Research  and  Development  (RED)  in  the 
private  and  governmental  sectors  is  evident  from  the  quan¬ 
tity  cf  information  available  in  professional  papers,  jour¬ 
nals,  and  books.  This  chapter  reviews  a  portion  of  this 
literature  which  concentrates  cn  BSD  project  selection  and 
resource  allocation  models,  the  suggested  utilization  of 
these  models,  their  limitations,  and  their  acceptability . 
The  purpose  of  this  review  is  to  investigate  demonstrated  or 
proposed  methods  of  solving  a  problem  similar  to  that  previ¬ 
ously  presented  and  should  not  be  viewed  as  an  exhaustive 
study  of  RED  literature. 

Several  published  articles  review  a  number  of  proposed 
methodologies  and  models  in  the  RED  project  selection  and 
resource  allocation  area.  These  review  articles  tend  to 
classify  model  forms  into  one  of  two  general  categories. 
Eaker  [Ref.  6]  titled  the  categories  as  "benefit  measure¬ 
ment"  and  "project  selection  and  resource  allocation." 
winkcfsky,  et  al.  [Ref.  7]  described  similar  classifica¬ 
tions  as  "value  measurement"  and  "portfolio  selection." 
Eenefit  or  value  measurement  models  rely  on  subjective 
expert  judgement  for  project  evaluation  combined  into  check¬ 
lists,  comparative  project  scores,  or  economic  indices. 
Project  or  portfolio  selection  and  resource  allocation 
models  normally  use  a  mathematical  programming  or  ether 
optimizing  technique  to  maximize  a  benefit,  such  as  profit, 
subject  to  constraints,  such  as  budget  and  manpower.  Other 
articles,  Augood  [Ref.  8],  Gear,  et  al.  [Ref.  9],  Gear 
[Ref.  10],  Newman  [Ref.  11],  and  other  authors  review  or 
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propose  models  that  can  be  classified  into  tnese  general 
categories.  Benefit  treasure  men  t  and  portfolio  selection  will 
te  used  for  model  classific aticn  in  this  review. 

Seme  authors,  Winkofsky,  et  al.  [Ref.  12], 
Eaker,  et  al.  [Ref.  13],  and  Allen,  et  al.  [Ref.  14], 
propose  methodologies  for  implementing  series  of  models  for 
hierarchical  applications.  In  addition  to  the  model  and 
methodology  proposal  and  review  articles,  Brandenburg 
[Ref.  15],  Baker  [Ref.  6],  Souaer  [Ref.  16  ani  17],  and 
other  authors  suggest  BSD  model  analysis  criteria  as  well  as 
observed  model  limitations  and  acceptability. 

E.  RESEARCH  AND  DEVELOPMENT  MODELS 

In  addition  to  the  general  classifications  of  benefit 
measurement  and  portfolio  selection,  models  are  further 
defined  ly  a  common  form,  such  as  an  index  model  or  a  linear 
programming  model,  a  description  of  that  form,  and  the 
recommended  or  observed  use  of  the  model. 

1  •  Eenefit  Measurement  Models 

Eenefit  measurement  models  typically  use  subjective 
evaluation  cx  a  number  of  R  SD  project  attributes  and  combine 
these  evaluations  into  a  project  value  or  worth.  These 
models  are  classified  into  the  checklist,  scoring  and  index 
model  forms. 

a.  Checklist  Models 

These  models  are  the  simplest  type  and  use 
subjective  comparisons  of  a  project  against  a  list  of 
elements.  The  elements  describe  project  criteria  or 
considerations,  established  by  the  management,  that  have 
some  significance  towards  success.  Augood  [Ref.  8]  suggests 
five  general  criteria  subdivided  into  53  specific  elements. 


Cooper  [Eef.  18]  surveyed  103  industrial  product  fims  an! 
was  atle  tc  analytically  reduce  his  48  similar  elements  into 
13  meaningful  factors.  Augood  [fief.  8]  also  suggests  rating 
projects  against  the  checklist  elements  by  using  descriptors 
such  as  very  unfavorable,  unfavorable,  average,  favorable, 
and  very  favorable.  The  worth  of  a  project  is  provided  by 
observing  the  number  cf  criteria  it  meets  or  how  "favorable" 
the  project  looks  by  its  overall  element  descriptor  ratings. 
This  method  then  reguires  judgemental  evaluations  cf  the 
checklist  results  for  each  project.  Winkofsky,  et  al. 
[Ref.  7  ]  in  their  literature  review,  observed  that  checklist 
models  are  most  suitable  for  project  evaluation  in  the 
exploratory  R5D  phase  when  only  preliminary  project  informa¬ 
tion  is  available. 

t.  Scoring  Kodels 

Scoring  models  are  quantified  extensions  of  the 
checklist  models  and  provide  a  numerical  score  or  value  for 
each  project.  Using  a  similar  criteria  list  as  used  in  the 
checklist  models,  the  project  score  is  typically  generated 
from  picducts  of  assigned  criteria  weights  and  project 
values  associated  with  criteria  accomplishment.  These  prod¬ 
ucts  are  then  summed  ever  all  criteria  to  provide  the  final 
project  score.  Winkofsky,  et  al.  [Hef.  7]  presented  the 
general  scoring  model  form  as 


PVj  =  Sum/i  [  (Wi)  ( Si j)  J 


{3.1) 


where  , 

i  =  criterion  number, 

j  =  project  number, 

PVj  =  jth  project  value  to  organization, 

Wi  =  ith  criteria  cr  element  weight,  and 

Sij  =  jth  project  accomplishment  of  the  ith  criterion. 


The  weights  such  as  7,i  or  Si  j  are  normally  assigned  fcy  F&D 
managers  or  F.ZD  advisory  groups  rased  on  personal  cr  group 
judgement-  Cooper  [Eef.  18]  empirically  derives  criteria 
weights  through  subjective  survey  data  and  a  multivariable 
regression.  Augood  [Sef.  8]  suggested  applying  the  weights 
C,  2,  5,  8,  and  10  to  his  five  increasing  benefit  element 

descriptors  and  subjectively  assigning  relative  importance 
weights  to  each  element.  He  then  represented  a  project 
score  as 

PVj  =  (100  Sum/i  [  ( »i)  {Si  j)  ]}/(10  Sum/i  [  Wi  ])  (3.2) 

where  the  previous  notation  of  equation  3.1  is  used.  This 
form  bounds  the  score  between  0  as  failure  to  100  as 
success.  Scoring  models  have  been  used  for  project  selection 
in  a  Eumter  of  organizations,  according  to  Winkofsky,  et  al. 
[Ref.  7],  with  various  criteria  and  weighting  schemes 
utili ze  d. 

c.  Index  Models 

These  models  directly  involve  probability  of 
project  success  and  costs.  The  general  form  of  an  index 
model  is; 

I  =  (P)  (E)  /  C  (3.3) 

where , 

I  =  project  index  value, 

P  =  project  probability  of  success, 

B  =  project  benefit,  and 
C  =  cost. 

The  differences  in  the  index  models  proposed  in  the  litera¬ 
ture  are  determined  by  what  is  included  in  the  three  model 


parameters.  The  probability  of  success  nay  include 
subfactcr  probabilities  such  as  success  in  research,  tech¬ 
nology,  production,  and  market  areas.  3enefits  include 
items  such  as  savings,  profits,  or  cash  flow.  Costs  range 
from  F6C  investment  tc  total  lifecycle  expenditures.  These 
indices  may  be  used  alone  or  combined  into  other  indices  to 
provide  the  final  prcject  measurement  such  as  the  Anscff 
index,  suggested  by  fiinkofsky,  et  al.  [Eef.  7]j  as  typical 
cf  RSE  project  selection  indices.  Ansoff's  index  provides  a 
numerical  value  called  the  Figure  of  Merit  (FM)  such  that 

FM  =  rdp  (  T  +  B  }  E  /  (Total  investment)  (3.4) 

where, 

r  =  probability  cf  research  success, 
d  =  probability  cf  development  success, 
p  =  probability  cf  market  success, 

1  =  index  of  technical  merit, 

B  =  index  of  business  merit,  and 
E  =  present  value  cf  earnings  from  project. 

like  the  scoring  models,  the  index  models  combine  a  number 
cf  prcject  attributes  into  a  single  value  which  may  then  be 
used  to  rank  projects  and  assist  in  determining  prcject 
selection.  More  cost  analysis  is  required  for  typical  index 
models  than  is  generally  necessary  for  scoring  models. 

2.  Portfolio  Selection  Models 

Ecrtfolio  selection  models  are  normally  more  compli¬ 
cated  than  benefit  measurement  models;  however,  they  also 
provide  additional  flexibility  and  realism  to  the  K  £  D  deci¬ 
sion  maker,  according  to  Souder  [Ref.  17].  Typically  these 
models  utilize  a  mathematical  programming  technique  to 
assist  in  project  selection  by  providing  the  resource 


allocation  that  will  maximi  ze  the  benefit  contriDuticn  for  a 
set  cf  possible  projects.  The  models  are  classified  within 
the  portfolio  selection  category  by  tne  mathematical 
programming  form  used.  Linear/  non-linear,  integer,  and 
dynamic  programming  model  forms  have  been  proposed  in 
literature.  Winkofsky,  et  al.  £Ref.  7]  observed  in  their 
review  that  these  portfolio  selection  models  assume  that  an 
explicit  benefit  or  contribution  function  can  be  determined. 
They  reviewed  other  authors ,'  however,  that  utilize  an  inter¬ 
active  programming  mcdel  form  which  assumes  an  implicit 
benefit  function  exists  but  is  never  formally  defined. 

a.  Linear  Programming 

The  general  form  of  the  linear  programming  model 


maximize 


(3.5) 


subject  to  Ax  <  b 
0  <  x  <  1, 

where , 

x  =  n-ccmponent  project  resource  allocation  vector, 
c  =  n-component  project  benefit  vector, 

A  =  mxn-ccmponen t  project  resource  utilization  matrix,  and 
b  =  m-component  available  resource  allocation  vector. 

Linear  programming  computations  provide  an  optimal  resource 
allocation  for  all  projects  provided  that  the  values  repre¬ 
sented  by  c,  b,  and  A  can  be  explicitly  obtained  and  the 
assumed  linear  relationships  between  resources  and  benefits 
portray  an  accurate  model  of  the  using  organization's  situ¬ 
ation.  The  project  resource  allocation  vector  x,  provided 
by  the  computations,  takes  values  between  and  including  zero 
and  cne.  Zero  represents  no  resource  allocated  on  that 


specific  project  ard  a  one  represents  full  allocation. 
Values  between  zero  and  one  provide  the  proportion  of  full 
resource  allocation  necessary  for  the  optimal  scluticn. 
Asher  £Ref.  19]  proposed  a  linear  programming  model  to  allo¬ 
cate  manpower  teams  involved  in  the  testing  operation  of  new 
chemical  compounds  UEder  development  by  a  pharmaceutical 
company.  The  project  resource  allocation  vector  represented 


the  number  cf  tests  cf  a  specific  compound  each  team  would 
conduct  and  was  bounded  by  the  total  number  of  each  chemical 
compound  available.2  The  expected  return  of  tne  compound  to 
the  firm  was  used  as  the  the  project  benefit  vector  and  was 
calculated  as  the  product  of  the  probability  of  success  and 
the  estimated  net  market  value  of  success  for  each  compound. 
The  project  resource  utilization  matrix  was  egual  to  the 
number  of  hours  necessary  for  each  team  to  test  a  specific 
compound  and  was  based  on  the  team's  experience  and  skill 
levels.  The  resource  vector  was  the  manhours  available  for 
each  team.  The  computation  results  provided  which  manpower 
team  should  test  which  compound  and  the  quantity  cf  that 
compound  they  should  test  in  order  to  maximize  the  expected 
value  of  return  to  the  company. 

b.  Non-linear  Programming 

Non-linear  programming  models  are  very  similar 
to  the  linear  models  but  attempt  to  represent  reality  as 
non-linear  relationships  rather  than  strictly  linear. 
Non-linear  models  are  formulated  with  similar  notation  as 
equations  3.5  and  provide  similar  resource  allocation  infor¬ 
mation.  A  non-linear  benefit  function  has  the  intuitive 
appeal  that  as  more  units  of  resource  are  provided  to  a 
project  less  benefit  is  returned  for  each  additional  input 

2The  bound  of  one  was  previously  noted  for  the  typical 
linear  program  model.  Asher's  computations  found  the  actual 
allocation  value  enstead  of  a  proportion. 
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unit.  Chidambaram  [Ref.  20]  considered  three  separate 
concave  nondecreasing  functions;  quadratic,  exponential,  and 
logarithaic;  which  all  approximated  his  U.S.  Army  data  of 
estimated  future  military  benefits  for  a  set  of  projects. 
Souder  [Eef.  16]  proposed  and  analyzed  four  portfolio  selec¬ 
tion  models  based  on  similar  models  in  the  literature  using 
thirty  actual  ROD  projects.  For  his  model  Souder  used  two 
types  of  ncn-linear  benefit  functions  which  are  represented 
by  Figures  3.1  and  3.2.  The  exponential  type.  Figure  3.1,  is 
also  siailar  to  Chidambar am ' s  benefit  functions.  Souder 
suggests  using  this  type  of  function  when  incremental 
resource  expenditures  in  the  lower  x  domain  are  expected  to 
greatly  increase  the  return,  and  the  benefit  diminishes  at 
some  point  as  resource  levels  are  increased.  The  S-shaped 
benefit  function.  Figure  3.2,  was  suggested  for  project 
cases  where  success  was  related  to  a  breakthrough  in  tech¬ 
nology.  In  this  case  the  return  from  the  resource  input  was 
initially  expected  to  be  lower  until  the  breakthrough.  Then, 
small  additional  resource  expenditures  returned  larger 
benefits  until  a  diminishing  return  point  is  reached. 


Figure  3.2  S-Shaped  Type  of  Benefit  Function. 


c.  Goal  Programming 

Goal  programming  is  a  mathematical  programming 
method  which  incorporates  several  objective  functions  as 
organizational  goals  and  attempts  to  solve  them  simultane¬ 
ously.  The  objective  or  goal  functions  become  the 
constraints  of  a  linear  or  ncn-linear  programming  problem. 
The  problem's  overall  objective  function  is  a  composite  of 
the  goal  functions  which  are  weighted  in  their  relative 
accomplishment  importance  by  the  program  user.  The  computa¬ 
tion  nirimizes  the  prioritized  deviations  from  the  goal 
constraints  as  described  by  the  composite  objective  function 
and  defines  a  ranee  of  feasible  solutions  between  the 
competing  goals.  Gcal  programming  in  R&D  models  is  normally 
used  with  other  model  forms  such  as  in  Taylor,  et  al. 
[Ref.  21]  and  Winkofsky,  et  al.  [Ref.  12]. 

d.  Integer  Programming  Models 

Integer  programming  models  are  formulated  in  a 
similar  manner  to  the  linear  and  non-linear  models,  however 
the  project  resource  allocation  vector  is  constrained  to  be 
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integers  only.  Taylor,  et  al.  [Bef.  21]  utilized  an 
integer  program  model  incorporating  goal  programing  methods 
to  allocate  thirty  researchers  among  seven  PSD  projects. 
Non-linear  constraints  were  used  to  relate  the  number  of 
researchers  assigned  to  a  project  to  probability  of  success, 
monetary  return,  ard  project  time  completion.  Linear 
constraints  were  used  to  describe  budget  limitations  and 
computer  capacity  utilization.  A  linear  composite  objective 
function  was  used  that  described  prioritized  deviations  from 
each  goal  constraint  and  the  integer  program  computation 
minimized  these  deviations  to  provide  the  optimal  allocation 
solution.  Other  authors  have  proposed  simpler  0-1  integer 
program  models  which  provide  a  go  or  no-go  project  decision 
depending  on  the  resource  allocation  vector  values  of  cne  or 
zero  respectively. 


e.  Dynamic  Programming  Models 


The  first  three  portfolio  selection  model  forms 


described  are  closely  related  and  differ  only  in  the  benefit 
and  constraint  functional  relationships  and  the  domain  of 
the  resource  allocation  vector.  A  dynamic  programming 
approach  to  BSD  project  selection  provides  a  different  opti¬ 
mizing  techr.igue  with  a  sequential  decision  process.  Hess 
[Ref.  22'  presents  such  an  approach  as  more  typical  of  the 
E8D  budgeting  problem  than  a  current-cycle-only  optimizing 
solution.  Hess  suggests  that  most  BSD  resource  allocation 
models  do  net  adequately  consider  the  periodic  re-evaluation 
cf  ESC  projects  which  stem  from  the  increase  in  information 
obtained  during  the  ESD  process  as  well  as  the  cyclic  budg¬ 
eting  evolution.  Hess  proposed  using  the  mathematical 
recursive  technique  cf  dynamic  programming  to  develop  an 
optimal  resource  allocation  policy,  ?o,  for  the  series  of 
subsequent  project  resource  decisions  stages.  The  general 


f  1  (X)  =  max  over  P  [  R1(x,P)  ] 


(3.6) 


for  the  last  decision,  and 

fn  (x)  =  max  over  P  [  En(x,P)  +  fn-1  [xn-1  (x,?)}  ]  (3.7) 

for  the  nth  prior  decision  where, 

fn  (x)  =  maximum  benefit  when  n  decisions  ramain, 

x  =  initial  quantity  of  resource, 

P  =  policy  cr  series  of  resource  decisions  used, 

xn-1(x,P)  =  resource  function  of  x  and  ?  at  stage  n-1,  ani 
Rn(x,P)  =  benefit  returned  in  stage  n  using  x  and  P. 

Assuming  En  (x, p)  is  known  through  some  benefit  measurement 
metncd,  the  optimal  policy,  Po,  is  calculated  ty  first 
solving  equation  3.6  for  the  last  decision,  stage  1.  Next, 
equation  3.7  is  solved  for  n=2  witn  the  substitution  of 
equation  3.6  The  value  of  n  is  increased  and  equation  3.7  is 
solved  again  with  the  substitution  of  the  n-1  results.  Each 
incremental  increase  in  n  represents  the  next  prior  decision 
stage  in  the  sequential  RSD  resource  allocation  process.  The 
dynamic  programming  model  is  dependent  on  knowing  the 
benefit  values  of  Ri(x,P)  (i  =  n,n-1 ,. . . ,  2, 1)  .  This  requires 
project  information  representing  the  entire  project  F.SD 
phase  and  the  number  of  cycles  remaining  until  project  RSD 
completion.  This  information  is  normally  based  on  subjective 
evaluations  and  estimates  which  are  used  to  derive  the 
resource  to  benefit  relationships.  Gear,  et  al.  [Ref.  9] 
analyzed  several  dynamic  programming  models  similar  to  and 
including  Hess’  model.  They  observed  that  this  form  of  the 
model  has  the  advantage  of  catering  to  the  multistage 
learning  and  decision  nature  of  RSD  projects.  Although  this 
aspect  is  referenced  throughout  the  literature,  the  model 
forms  presented  above  do  not  consider  it  as  well  as  the 
dynamic  programming  form. 


f.  Interactive  Programming  Models 

Winkofsky,  et  al.  [Ref.  7]  reviewed  several 
authors  that  proposed  interactive  programming  models  as 
solution  techniques  for  multiple  criteria  problems.  These 
models  can  provide  helpful  tools  in  the  R5D  portfolio  selec¬ 
tion  process.  Zoints  and  Wallenius  [Ref.  23]  proposed  a 
man-machine  interactive  programming  method  which  allows  a 
decision  maker  to  optimize  an  implicit  benefit  function 
involving  multiple  objective  or  goal  functions.  The  benefit 
function  is  a  composite  objective  function  and  is  unknown  to 
the  decision  maker;  however,  it  is  assumed  to  be  a  linear 
function,  or  in  a  mere  general  form  a  concave  function,  of 
several  known  goal  functions.  The  implicit  composite  objec¬ 
tive  function  is  maximized  against  a  set  cf  convex 
constraints  through  the  decision  maker's  answers  to  yes-or- 
no  guesticr.s  involving  trade  offs  between  possible  goal 
function  solutions.  Zoints  and  Wallenius'  method  initially 
optimizes  a  computer  generated  composite  objective  function 
including  the  goal  function  constraints  and  an  arbitrary  set 
of  function  multipliers.  The  technigue  produces  a 
Paretc-opti nal  solution  to  the  problem.  Next  a  subset  of 
nonbasic  variables  is  generated  that,  if  introduced  into  the 
basis,  would  continue  to  yield  a  Pareto-optimal  solution. 
For  these  selected  variables  a  series  of  trade  offs  are 
defined  which  increase  or  decrease  each  of  toe  goal  func¬ 
tions  and  the  trade  offs  are  presented  to  the  decision 
maker.  The  decision  maker's  responses  to  the  trade  offs  are 
used  to  generate  a  new  set  of  goal  function  multipliers.  The 
new  multipliers  develop  a  new  composite  objective  function 
closer  to  the  implicit  function.  The  authors  state  that 
successive  iterations  of  this  process  assures  convergence  to 
the  optimal  solution.  Their  premise  is  that  all 
Pareto-optimal  solutions  form  a  subset  of  all  extreme  point 
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solutions  ar.  1  with  each  iteration  a  Pareto-optimal  solution 
is  eliminated  from  the  set  of  possible  optimal  solutions. 
Since  there  is  a  finite  number  of  extreme  points  there  will 
be  a  finite  number  of  iterations  required  to  obtain  the 
optima]  solution. 

C.  DECISION  PROCESSES 

The  majority  of  models  presented  in  the  literature  are 
proposed  for  or  utilised  in  single  level  organizations. 
Several  authors  have  proposed  extending  these  model  forms 
into  methodologies  or  decision  processes  for  use  by  multi¬ 
level  organizations.  Baker,  et  al.  [Ref.  13]  and 

Winkofsky,  et  al.  [  Bef .  12]  presented  such  processes.  Beth 
articles  concentrated  on  budget  allocation  of  RSD  dollars  in 
a  multilevel  organization  and  relied  on  the  interlevel 
communication  and  interaction  to  complete  the  process, 
flinkofsky,  et  al.  [Ref.  12]  proposed  a  decision  process 
model  in  which  many  subunits  within  a  three  level  organiza¬ 
tion  are  considered.  Each  unit  attempts  to  minimize  the 
deviations  from  its  0-1  goal  constraints.  The  lowest  level 
formulates  and  solves  its  goal  program  and  provides  the 
resulting  solution  to  the  middle  level.  The  middle  level 
formulates  a  goal  program  incorporating  ail  subordinate  unit 
program  results  and  additional  constraints  from  the  higher 
level.  In  turn  the  higher  level  solves  a  composite  goal 
program  incorporating  middle  level  results.  Conflicts 
between  the  levels  are  resolved  either  through  communication 
between  levels  or,  if  necessary,  as  directed  by  higher 
levels.  The  process  iterates  to  an  optimal  solution. 

E.  ANALYTIC  HIERARCE3  PROCESS 

Saaty  [Ref.  24]  proposes  the  "Analytic  Hierarchy 

Process"  (AHP)  as  a  decision  tool  for  determining  priorities 


and  making  trade-offs.  Saaty  provides  many  possible  applica¬ 
tions  for  this  process  of  which  F.SD  resource  allocation  is 
one.  Through  structuring  a  hierarchy  of  criteria  stake¬ 
holders  and  outcomes  and  by  developing  judgemental  based 
priorities  the  process  contributes  in  complex  problem 
solving  and  predictions  of  likly  outcomes.  AHP  uses  three 
principals  of  analytic  thinking;  structuring  hierarchies, 
setting  priorities,  and  logical  consistency.  AH?  enables 
consideration  of  a  problem  as  a  whole  and  the  study  cf  the 
component  interactions  within  a  Hierarchical  problem 
description.  Saaty  describes  an  eight  step  process  which 
encompasses  a  graphical  breakdown  of  the  problem  into  a 
decision  tree  hierarchy,  involves  pairwise  criteria  compari¬ 
sons,  establishes  decision  priorities,  and  evaluates  the 
consistency  of  the  comparisons.  Figure  3.3  shows  his  example 
hierarchy  structure  for  choosing  fi&D  projects  to  ensure 
adeguate  future  power  and  electricity.  The  figure  lists  six 
levels  of  hierarchy,  one  focus  or  central  objective  and  the 
remaining  levels  each  with  several  elements.  The  process 
involves  a  pairwise  comparison  starting  from  the  top.  A 
higner  level  is  used  as  the  comparison  property  for  the  next 
lower  level.  The  pairwise  comparisons  of  the  elements  under 
a  property  become  the  basis  for  the  computation  of  relative 
element  priorities.  Finally,  the  consistency  of  the  compari¬ 
sons  is  investigated. 

E.  MCEE1  ACCEPTABILITY  AND  ANALYSIS 

The  acceptability  of  R&D  project  selection  models  by 
industry  and  government  has  leng  been  a  problem.  This  is 
evident  from  statements  in  the  majority  of  articles 
reviewed.  Baker  [Ref.  6]  lists  seven  limitations  of  proposed 

F. 6D  models  based  on  his  and  others'  research.  Baker's 
summary  cf  FED  model  limitations  is  as  follows: 
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1)  Inadequate  treatment  cf  project  and  parameter 
interrelations  with  respect  to  Lota  Denefit  contribution 
and  tc  resource  utilization. 

2)  Inadequate  treatment  or  uncertainty  as  it  impacts  in 
benefit  measurement  and  parameter  estimation. 

3)  Inadequate  treatment  of  multiple,  interrelated  deci- 
sicn  criteria  which  have  no  underlying  measure. 

4)  Inadequate  treatment  of  the  time  variant  property  of 
the  parameters  and  criteria  and  the  associated  problem 
of  ccntinuity  in  the  research  program  and  staff. 

5)  A  restricted  view  of  the  problem  which  (a)  portrays  a 
once-a-year  investment  decision  rather  than  an  intermit¬ 
tent  stream  of  investment  alternatives,  (b)  does  not 
include  such  attributes  as  timing  of  the  decision, 
generation  of  additional  alternatives,  and  recycling, 
(c)  dees  not  recognize  the  diversity  of  projects  along 
the  spectrum  from  basic  research  to  engineering,  and  (d) 
views  the  problem  as  a  decision  event  rather  than  a 
hierarchical,  diffuse  decision  process. 

6)  Nc  explicit  recognition  and  incorporation  cf  tne 
importance  of  individual  RSD  personnel. 

7)  The  inability  to  establish  and  maintain  balance  in 
the  RSE  program;  e.q.,  balance  between  basic  and  applied 
research,  between  offensive  and  defensive  research, 
between  breakthrough  and  improvement  orientations, 
between  in-house  and  contracted  projects,  between 
product  and  process  oriented  projects  and  between  high 
risk/high  payoff  and  low  risk/moderate  payoff  projects. 


Eaker  continues  with  the  conclusion  that  this  list  cf  limi¬ 
tations  makes  it  clear  why  few  RSD  models  have  been  imple¬ 
mented  fer  use  by  FSD  managers  and,  in  the  cases  where 
models  are  utilized,  the  majority  are  the  simpler  scoring  or 
index  form. 

Scuder  [Ref.  17]  developed  a  scoring  model  to  assess  the 
suitability  of  RSD  models.  His  criteria  used  for  the  model, 
based  on  his  interviews  with  management  scientists  and  RSD 
administrators,  are  presented  in  Table  II.  Souder  then  used 
his  scoring  model  tc  rate  26  models  for  use  in  RSD  project 
selection  decision  support.  Index  and  scoring  model  forms 
were  again  found  tc  be  in  greater  use  than  other  model 
forms . 
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TABLE  II 

Five  Model 

Criteria  and  Their  Characteristics 

Criteria 

Cha  racteristics 

REALISM 

model  includes: 

Multiple  Objectives 

Multiple  Constraints 

Market  Ris*.  Parameter 

Technical  Risk  Parameter 

Manpower  Limits  Parameter 

Facility  Limits  Parameter 

Budget  Limits  Parameter 

Premises  Uncertainty  Parameter 

FLEXIBILITY 

model  applicable  to: 

Applied  Projects 

Basic  Projects 

Priority  Decisions 

Termination  Decisions 

Initiation  Decisions 

Budget  Allocation  Application 

Project  Funding  Application 

CAPABILITY 

models  performs: 

Multiple  Time  Period  Analyses 
Optimization  Analyses 

Simulation  Analyses 

Scheduling  Analyses 

USE 

model  is  character ized  by  : 

Familiar  Variables 

Discrete  Variables 

Computer  Not  Needed 

Special  Persons  Not  Needed 

Special  Interpretations  Not  Needed 

CCST 

model  has: 

Low  Set-up  Costs 

Low  Personnel  Costs 

Low  Computer  Time 

Low  Data  Collection  Costs 

F.  S08MAR Y 

Ihis  chapter  has  reviewed  several  BSD  project  selection 
model  forms  found  through  cut  the  li terat ure .  The  models 
presented  were  typical  of  the  literature  and  were  the  more 
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general  ions  which  ccuid  have  application  in  the  following 
analysis.  The  commonality  observed  between  model  forms  is 
the  use  of  subjective  project  benefit  relationships  ir.  seme 
manner.  Decision  processes  and  the  Analytic  Hierarchy 
frocess  were  briefly  presented  as  methods  for  approaching 
the  R£D  project  selection  problem.  Table  III  summarizes  the 
models  presented  in  this  chapter.  Other  proposed  project 
selection  models  are  listed  in  the  bibliography.  The  next 
chapter  describes  a  R&D  program  prioritization  process 
proposed  within  the  Marine  Corps,  and  classifies  the  process 
model  form  according  to  the  literature. 
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TABLE  III 

Summary  of  Reviewed  Models  and  References 


1 


Authci/Re fere nee 

Asher  M962) 
[Eef.  TS" 


[Eef.  t?: 
Augood^  ^1973) 


Baker  / 1  574 } 
[Eef.  6] 


Chid antaram  (1970) 
[Eef.  20] 

Cccrer  (1981) 

[  R  er  .  18] 

Hess  (1  962) 

[ Eef.  22  ] 

Seeder  (1973) 

[Eef.  16] 


Taylor  et  al.  (1982) 
[Eer.  21 j 

Winkofsky  et  al.  (1980) 
[Ref.  7 r 


Winkofsky  et  al.  (1981) 
[Ref.  12. 

Zcints  et  al.  (1  976) 
[Eef.  23] 


USSSi  Form  Descriptor  Dsed 
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Quantified 
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Indices 

Eenefit  Measurement 
Comparative 
Scoring 

Benefit  Contribution 
Project  Selection/Allocation 
Ranking 

Linear  Programming 
Non-linear  Programming 
Dynamic  Programming 
Integer  Programming 

Ncn-linear 


Empirical  (scoring) 
Dynamic  Programming 


Profitability  Index 
linear 
Non-linear 
Zero-One 

Integer  Ncn-linear 
Goal  Programming 

Value  Measurement 
Checklist 
Scoring 

Economic  Index 
Portfolio  Selection 
Linear  Programming 
Integer  Programming 
Nonlinear  Programming 
Interactive  Programming 

0-1  Goal  Programming 


Interactive  Program  mi rg 
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IV.  MCEEC  PRICRITIZATION  PROCESS 

A.  IHTBCDOCTION 

The  MCDEC  Prioritization  Process  consists  of  four  tire- 
dependent  phases  synchronized  with  the  Marine  Corps  partici¬ 
pation  in  the  DOD  Planning  Programming  and  Budgeting  System 
and  is  organized  fee  implementation  at  the  Marine  Corps 
Development  Center  (EevCtr)  ,  MCDEC.  Figure  4.1  depicts  this 
synchronization  and  the  process  timetable  for  a  single 
cycle.  The  four  phases  include  determining  deficiencies 
and  reguirements,  prioritizing  deficiencies  and  require¬ 
ments,  prioritizing  research  and  development  programs,  and 
completion  of  funding  profiles.  This  process,  as  described 
fcy  Major  J.  L.  Creed,  USMC,  in  "A  Guide  for  the  Performance 
of  the  Development  of  the  MCDEC  RSD  Program  Prioritization 
Process,  Methodology  Manual"  £Ref.  25],  is  carried  out  under 
the  responsibility  of  the  Office  of  the  Deputy  Chief  of 
Staff  (EC/S),  Developmental  Coordination  (DC),  DevCtr, 
MCDEC,  and  ty  two  primary  groups,  the  Prioritization  Working 
Group  (FRWG)  and  the  Chief  of  Staff's  (C/S’s),  DevCtr 
Frioritization  Committee  (Pri  Com) .  Each  Division  and 

Directorate  of  the  EevCtr  provides  a  representative  for 
membership  in  the  PRWG.  The  Division  and  Directorate  Heads 
compose  the  C/S  1  s  DevCtr  Prioritization  Committee.  Appendix 
A  details  the  phases  and  tasks  of  the  process  and  lists  the 
EevCtr  Divisions,  Directorates,  and  Developmental 

Coordination  Branches  responsible  for  completing  each  step. 
This  chapter  describes  the  process  concepts  and  concentrates 
on  the  participants  as  shown  in  Figure  4.2  . 
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Figure  4.  1  MCEEC  Process  and  BDT&E  POM  Cycle. 
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Figure  4.2  flCDEC  Frocess  Participants. 


E.  EECCESS  PHASES 
1 .  Fhase  I 

The  determination  of  deficiencies  and  requirements, 
phase  I,  involves  three  general  tasks  and  serves  to  provide 
the  FFWG  with  an  information  base  for  the  following  subjec¬ 
tive  evaluations.  First,  the  Marine  Corps  Midrange 
Objectives  Flan  (MMROE)  and  the  Marine  Corps  Long  Range  Plan 
(MLR F )  are  updated  utilizing  Defense  Guidance  (DG) ,  the 
FYDP,  previous  FOM  recommendations,  and  other  appropriate 
research  and  development  information. 


Second,  Fleet  Marine  Force  (FMF)  input  is  solicited 
ly  survey  analysis  through  questionnaires,  opinicnnair es, 
and  personal  interviews.  This  task  provides  the  eventual 
system  users  with  an  opportunity  to  input  observations  of 
mission  def iciencies,  ideas  for  new  programs,  and  their 
rankings  of  ongoing  programs. 

Finally,  MAAs  are  conducted  which  provide  the  frame¬ 
work  necessary  for  phase  II  and  serve  to  formalize  all  defi¬ 
ciencies  that  diminish  the  capability  to  perform  the  mission 
area.  The  MAAs  are  classified  according  to  the  Mission  Area 
Elements  (MAEs)  listed  in  Table  I,  and  performed  by  project 
officers  under  the  direction  of  the  Planning  and  Evaluation 
Eranch,  DevCtr.  The  assigned  project  officer  conducts  his 
analysis  through  a  series  of  mission  categorizations  which 
are  based  or.  the  previously  updated  plans  and  other  estab¬ 
lished  Marine  Corps  Ecctrine.  Three  basic  categorizations 
occur  during  the  MAA.  First,  the  mission  area  element  is 
defined  into  its  general  and  specific  operational  mission, 
then  the  resources  necessary  to  accomplish  the  operational 
missions  are  identified,  and  finally  each  resource  is 
analyzed  against  each  operational  mission  to  identify  if  any 
deficiencies  exist.  The  collection  of  these  identified 
mission  deficiencies  present  the  recommended  corrective 
actions  to  obtain  full  Marine  Corps  capability  in  a  mission 
area . 

2  .  Eh ase  II 

Phase  II  merges  the  identified  mission  element  defi¬ 
ciencies  into  a  single  prioritized  list.  To  accomplish  this 
ranking,  the  PHWG  first  receives  briefings  concerning  the 
MAAs  and  survey  analyses  from  phase  I.  Next,  the  FA«3 
constructs  a  decision  tree  based  on  tne  presented  informa¬ 
tion  to  order  deficiencies  within  mission  area  elements. 
The  tree  development  is  represented  by  Figures  4.3  through 
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Figure  4.5  Decision  Tree,  Besources. 


The  resources  shown  in  Figure  4.5  are  fu 
defined  into  principal  and  ancillary  categories.  Prin 
resoures  correspond  to  equipment  or  weapon  systems  tha 
used  to  accomplish  the  specific  mission  directly, 
ancillary  resources  are  necessary  but  only  indir 
support  the  mission.  A  common  list  of  resources  necessa 
accomplish  the  specific  operational  mission  is  init 
applied  to  all  specific  operational  missions  with 
resources  subsequently  being  determined  not  applicable 
all  missions.  As  an  example.  Creed  [Ref.  25]  sugge 
possible  resource  list  for  the  close  combat  mission 


rther 
cipal 
t  are 
and 
ec  tly 
ry  to 
ially 
seme 
for 
sts  a 
a  tea 


46 


Figure  4.6  Decision  Tree,  Deficiencies. 


element  to  le  direct  fire,  assault,  and  anti-armor  weapons 
for  the  principal  resource  category  and  communication  equip¬ 
ment  and  target  acquisition  devices  for  the  ancillary 
resources.  The  resource  deficiencies  and,  thus,  the  mission 
deficiencies  shown  in  Figure  4.6  are  identified  by  evalu¬ 
ating  each  resource  as  it  supports  the  accomplishment  of  the 
specific  operational  mission. 

The  PR WG  assigns  subjective  importance  values3  (0.01 
to  1.00),  to  each  branch  of  the  decision  tree  from  the 
mission  area  through  the  deficiency  level.  These  values  are 
based  on  presented  information  and  finalized  through  collec¬ 
tive  group  judgement.  The  Mission  Area  Values  (MAV)  repre¬ 
sent  the  relative  importance  of  each  mission  area  within  the 
OS MC  mission  and  the  Mission  Area  Element  Values  (MAEV) 
represent  the  relative  importance  of  each  mission  area 
element  within  the  mission  area.  Similarly,  General 
Operational  Mission  Values  (GOMV)  ,  Specific  Operational 
Mission  Values  (SOMV),  and  Deficiency  Values  (DV)  represent 
the  relative  importance  of  each  general  operational  mission 

3Thes<?  yalues  represent  linear  relative  importance,  (ie, 
a  .74  deficiency  is  twice  as  critical  as  a  .37  deficiency). 


within  the  mission  area  element,  eacn  specific  operational 
mission  within  the  general  operational  mission,  and  each 
deficiency  within  the  specific  operational  mission  respec¬ 
tively.  The  initial  value  sets,  MAEV,  GOMV,  SO  MV ,  and  DV, 
are  normalized  to  a  common  base  for  each  set  which  allow 
comparisons  across  all  elements,  missions,  and  deficiencies 
in  their  respective  decision  tree  level.  The  Deficiency 
Relative  Importance  Weight  (DRIW)  for  each  specific  mission 
deficiency  is  defined  as 

DRIW  =  MAV  •  MAEV  •  GOMV  •  SCMV  •  DV  { 4 . 1 j 

where , 

MAV  =  Mission  Area  Value, 

MAEV  =  Mission  Area  Element  Value, 

GCMV  =  General  Operational  Mis*sion  Values, 

SCMV  =  Specific  Operational  Mission  Values,  and 
DV  =  Deficiency  Value. 

The  DBJWs,  in  turn,  determine  a  mission  deficiency  rank 
ordering. 

3.  Phase  III 

Phase  III  prioritizes  ongoing  RDTSS  acquisition 
programs  through  presentation  of  the  prioritization  group 
and  committee  with  available  information,  assigning  program 
subjective  values,  deriving  program  effectiveness,  combining 
previous  program  pricritiza tions,  and  presenting  the  results 
for  Director,  DevCtr  approval.  First,  the  Prioritization 
Workirg  Group  receives  briefs  by  the  Developmental  Project 
Officers  (DPOs)  to  assist  in  determining  the  worth  and 
utility  of  the  program.  Next,  survey  analysis  briefings 
provide  the  FMF  evaluations  of  current  RDT&E  programs.  This 
information  presented  in  the  PRWG  briefs  is  also  provided 
separately  to  the  DC/S  DC  Prioritization  Committee. 
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The  next  task  cf  phase  III  is  the  subjective  priori¬ 
tization  cf  RDTSZ  programs.  D ur in g  this  task,  a  sample  of 
experts  are  solicited  for  their  desired  RDTSZ  program  rank 
orderings  and  relative  worth  values  (1.01  to  1.00}  .  This 
opinion  sample  consists  cf  the  DC/S  DC  Prioritization 
Committee  membership,  minimally,  and  may  include  other  qual¬ 
ified  individuals.  The  prioritization  is  performed  using  an 
analytic  Delphi  process  where  the  independent  rank  ordering 
and  worth  values  are  combined  and  returned  to  the  sample 
membership  for  study  and  re submissi on .  After  at  least  three 
replications  with  relatively  common  consecutive  listings, 
the  prioritization  concludes  with  a  program  ranking  and 
program  Subjective  Evaluation  Values  (SEVs)  . 

Following  the  subjective  evaluation  the  EDTSE 
programs  are  prioritized  based  on  their  mission  deficiency 
effectiveness.  The  EEWG  identifies  all  deficiencies  within 
each  mission  element  that  are  directly  affected  by  a  EDTSZ 
program,  and  it  evaluates  the  proportion  at  which  the 
program  accomplishes  the  specific  deficiency.  This  propor¬ 
tion  is  defined  as  the  Program’s  Proportion  of  Deficiency 
Accomplishment  (PPDA).  The  evaluation  process  relies 
heavily  cn  the  input  from  the  EPOs  concerning  each  program’s 
proposed  operational  characteristics.  The  product  of  the 
specific  DEIW,  assigned  in  phase  II,  and  the  specific  PPDA 
are  summed,  over  all  deficiencies,  to  obtain  a  program 
Deficiency-Derived  Efficiency  Value  (DDE V)  as 


DDEV  =  Sum/d  [  (DR  IK)  •  (PPEA)  ] 


(4.2) 


where , 


Sum/d  =  Summation  ever  all  deficiencies, 

DEIW  =  Deficiency  Relative  Importance  Weight  and 

PPEA  =  Program's  Proportion  of  Deficiency  Accomplishment 


The  programs  are  then  ranked  according  to  the  DDEV  values. 


The  PRWG  obtains  a  final  ED TS S  program  ranking  by 
combining  the  subjective  and  deficiency-derived  prioritiza¬ 
tions.  A  Subjective  Evaluation  List  Weight  (SELW)  and  a 
Eef iciency-Eeri ved  List  Weight  (DDLW)  is  assigned  to  the 
appropriate  program  list  combining  them  in  a  linear  rela¬ 
tionship  based  on  PB5G  consensus  of  each  list's  validity. 
The  sum,  by  program,  of  the  weighted  prioritized  lists 
provides  the  Final  RIT&E  Program  Value  (FPV)  as 

F? V  =  (SELW)  •  (SEV)  +  (DDLW)  •  (DD£V)  (4.3) 

where , 

SEIW  =  Subjective  Evaluation  List  Weight, 

SEV  =  Subjective  Evaluation  Value, 

DE1W  =  Deficiency-Ierived  List  Weight,  and 
EDEV  =  Deficiency-Derived  Efficiency  Value. 

The  final  prioritized  ranking  is  accomplished  according  to 
the  FEV  and  this  completes  phase  III. 

4 .  Eh use  TV 

Ehase  IV  completes  the  annual  process  by  sumirar izing 
the  three  preceding  phases  and  proposing  funding  profiles 
which  reflect  the  prioritized  program  ordering.  The  process 
summary  is  used  as  a  turnover  file  for  the  next  prioritiza¬ 
tion  cycle  and  the  funding  profiles  are  submitted  to  H1J1C 
for  utilization  in  the  RDTSE  PCM  development  process.  Creed 
[Ref.  25]  suggested  further  development  in  this  phase  which 
has  net  yet  taken  place.  Currently,  these  funding  profiles 
are  developed  based  on  solely  the  judgement  of  the  tasked 
EevCtr  branch  using  no  mathematical  optimization  or 
algorithms. 
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c. 


FCHM  Cl  MODEL  IN  SCDEC  EEOCESS 


Throughout  the  literature,  as  reviewed  in  the  previous 
chapter,  and  in  the  following  analysis  tne  form  of  a  budget 
allocation  and  program  selection  model  is  important.  The 
MCDEC  model  as  described  is  a  benefit  measurement,  scoring 
model.  As  with  typical  models  of  this  type,  the  MCDEC  model 
combines  various  attributes  into  a  single  value.  Kithin 
this  model  form  classification  differences  in  subjective 
benefit  weight  assignment  schemes  are  observed.  These 
differences  range  between  weight  assignment  by  a  single 
manager  to  assignment  based  on  a  multivariable  regression  cf 
subjective  program  evaluations  from  many  solicited  sources. 
The  KCEEC  process  attempts  to  base  its  benefit  weight 
assignmer.t  on  many  inputs  from  operational  and  ncn- 
cperaticnal  forces  as  well  as  agencies  outside  of  the  Marine 
Corps.  The  inputs  are  combined  into  benefit  weights  througn 
a  grcup  consensus  procedure.  In  general  the  MCDEC  prioriti¬ 
zation  process  is  similar  to  E5D  project  selection  models 
proposed  in  the  literature  and  is  classified  with  the 
majority  cf  implemented  models  in  industry. 

E.  SDMMABI 

This  chapter  has  presented  the  MCDEC  prioritization 
process  as  described  by  Creed  [Eef.  25]  and  by  information 
obtained  through  liaison  with  the  process  developing  agency, 
tne  Analysis  Support  Eranch,  DC/S  DC,  DevCtr.  The  process 
emphasizes  synchronization  with  the  Marine  Corps  participa¬ 
tion  in  the  DoD  PPBS,  and  utilization  of  ESD  program  evalua¬ 
tion  input  from  many  organizational  levels.  The  process 
requires  several  subjective  evaluations  concerning  the  ESD 
programs  and  mission  deficiencies  which  are  then  combined 
into  a  basis  for  RSE  program  ranking.  Table  IV  summarizes 
the  subjective  parameters  necessary  for  the  process.  The 


chapter  concludes  with  classifying  the  model  form  used  in 
the  MCE EC  process  as  a  benefit  measurement  scoring  model, 
according  tc  similarities  with  proposed  models  in  the  liter¬ 
ature. The  literature  suggests  ether  model  form  applications, 
model  limitations,  and  model  evaluation  criteria  which  will 
assist  further  investigation  cf  possible  improved  alterna¬ 
tives  cr  enhancements  to  the  MC3EC  prioritization  process. 


TABLE  IV 

Sub  ject ive 

Model  Parameters 

PHASE 

TASK 

PARAMETER 

ASSIGNMENT 

RANGE 

II 

3 

HA  V 

FF.WG 

0.01-1.00 

II 

3 

MAEV 

PR  W  G 

0.01-1.00 

II 

3 

GO  H  ^ 

PRWG 

0.01-1.00 

II 

3 

SO  M  V 

PE  W  G 

0. 01-1 .00 

II 

3 

DV 

PR  WG 

0.01-1.00 

III 

3 

SEV 

Pri  Com 

0.01-1.00 

III 

4 

PPI A 

FEW  G 

0.0  1-1.00 

III 

5 

SEIS. 

PR  WG 

NS 

III 

5 

DDLS 

ERWG 

NS 

NS 

=  Not 

S  pecified 

Ta 

sks  are 

detailed  in 

Appendix  A 

V.  A  MCDEC  PORTFOLIO  SELECTION  MODEL 

A.  IKTECDOCTION 

The  MCCEC  process  presented  in  the  previous  chapter  was 
identified  as  a  benefit  measurement  scoring  model  with 
similar  characteristics  as  models  currently  used  in 
industry.  The  literature  also  proposes  portfolio  selection 
models  which  use  benefit  measurement  relationships  to  opti¬ 
mize  resource  allocation  through  various  mathematical 
programming  techniques.  The  MCDEC  process,  as  proposed, 
does  net  attempt  to  optimize  program  resource  allocation 
and,  thus,  the  obvious  first  process  improvement  tc  be 
investigated  is  the  extension  of  the  current  model  into  a 
portfolio  selection  model.  The  simplest  and  most  comprehen- 
dable  way  This  can  be  accomplished  is  by  formulating  the 
process  as  a  linear  programming  model.  This  model  fern 
approximates  resource  and  benefit  relationships  with 
straightforward  linear  functions  that  are  easly  understood. 
The  linear  programming  model  is  the  basic  structure  for 
ether  more  complicated  portfolio  selection  models  and  will 
provide  a  base  for  observing  advantages  from  a  process  model 
change.  The  primary  purpose  of  the  following  linear  program 
development  is  tc  illustrate  the  feasibility,  flexibility, 
and  required  assumptions  of  a  portfolio  selection  model  as 
applied  to  the  MCDEC  prioritization  process  and  does  not 
include  all  constraints  and  relationships  required  fer  a 
complete  F.DT&E  funding  allocation. 

E.  FCRflOLATION 

The  initial  step  of  any  linear  programming  formulation 
can  be  viewed  as  a  thought  exercise.  In  this  exercise  the 


linear  program  is  viewed  as  a  black  oox  which  provides  an 
output  when  given  the  necessary  data.  The  thought  exercise 
defines  the  desired  output  and  the  remaining  formulation 
structures  the  linear  program  to  obtain  this  output.  The 
currently  proposed  HCDEC  process  output  provides  a  priori¬ 
tized  pregram  listing  based  on  relative  program  importance 
in  accomplishing  mission  deficiencies.  A  linear  programming 
output  which  represents  the  proportion  of  a  program  that  is 
selected  or  funded  provides  program  prioritization  and, 
additionally,  resource  allocation  information. 

The  next  formulation  step  is  to  define  index  nomencla¬ 
ture  for  the  element  categories  under  consideration  ir.  the 
linear  program.  The  MCDEC  process  concentrated  on  two 
elements,  mission  deficiencies  and  P.DTSE  programs.  These  two 
are  alsc  used  in  the  following  linear  programming  model 
formulations.  The  indexing  letter  i  will  represent  a 
program  and  the  letter  j  will  represent  a  mission  defi¬ 
ciency.  Also,  when  considering  a  resource  allocation 
process  several  time  periods  are  normally  necessary.  The 
letter  k  will  be  used  to  index  a  time  period  in  the  formula¬ 
tion.  The  total  number  of  programs,  deficiencies,  and  time 
periods  will  be  represented  by  the  letters  p,  d,  and  t 
respectively.  In  the  following  discussion  the  series  summa¬ 
tion  cf  any  indexed  values  will  be  denoted  as  Sum/index 
[indexed  values]  such  as  Sum/i[ Xi  ],  which  represents  the 
summation  cf  the  values  Xi  over  i  =  1...p,  where  p  is  the 
total  number  of  programs  under  consideration. 

The  benefit  measurement  output  data  provided  by  the 
ffCDEC  process  is  defined  in  the  previous  chapter  and  is  used 
here  for  the  linear  program  formulation.  Four  value  sets  of 
interest  are  developed  in  the  MCDEC  process.  The  Deficiency 
Relative  Importance  Weight  (DRIW)  is  established  in  phase 
II.  In  phase  III  the  Prioritization  Working  Group  develops 
the  Program’s  Proportion  of  Deficiency  Acs omplishment 


(PPEA)  /  and  the  Subjective  Evaluation  Value  (SEV).  These 
three  values  are  combined  in  equations  4.2  and  4.3  to 
provide  the  Final  E DTSE  Program  Value  (F2V)  .  Resource  data 
is  r.ct  defined  in  the  current  process  but  is  considered 
during  Project  Development  Officer  briefs  to  the  prioritiza¬ 
tion  group  and  committee.  The  resource  data  is  also  used  in 
phase  IV  to  provide  suggested  RDT&Z,Navy  PCM  input.  Previous 
FOMs  that  are  independent  cf  the  process  are  used  as  proxy 
resource  estimators  for  illustration  in  the  following 
formulations. 

The  decision  variables  for  the  formulations  are  implied 
from  the  desired  output.  The  following  series  of  linear 
programs  will  be  formulated  using  the  decision  variable  Xi 
which  represents  the  proportion  of  the  ith  program  that  is 
funded  or  Xik  which  represents  the  proportion  of  the  ith 
program  that  is  funded  in  the  kth  time  period.  The  values 
Xi  and  Xik  are  defined  between  and  including  zero  and  one. 
A  variable,  Yi,  which  represents  the  proportional  deviation 
from  a  fixed  jth  deficiency  accomplisnmen t  goal  is  used  in 
later  linear  programming  formulations  and  is  also  defined 
between  and  including  zero  and  one. 

As  in  the  literature,  the  objective  functions  for  the 
following  formulations  are  provided  from  the  benefit  meas¬ 
urement  model.  For  the  initial  linear  program  formulation 
the  Final  Program  Value  (FPV)  vector  from  the  MCDEC  process 
is  used.  This  vector  represents  a  combination  of  deficiency 
derived  and  subjectively  assigned  mission  deficiency  accom¬ 
plishment  program  values.  The  vector  FPV  contains  a  value 
FPVi  for  each  program,  where  i  is  an  integer  from  1  to  p. 
Formulation  5.1  provides  an  initial  unconstrained  linear 
programming  model  that  is  eguivalent  to  the  MCDEC  benefit 
measurement  model  and  can  be  solved  by  observation.  This 
linear  program  maximizes  the  total  relative  program  impor¬ 
tance  as  individually  defined  by  each  FPVi,  reduced  by  the 


available  program  funding  proportion  Xi,  and  summed  ever  all 
programs.  The  optiiral  solution  for  this  for mulat icr.  is 
obtained  when  Xi  is  set  at  one  for  all  i.  The  initial  formu¬ 
lation  is 


maximize  Sum/i  [  (FPVi)  (Xi)  ] 


(5.1) 


subject  to:  0  <  Xi  <  1 

i  =  1 .. .  p. 

Formulation  5.1  may  be  trivial  but  it  provides  a  linear 
program  structure  equivalent  to  the  MCDEC  process  which  can 
be  constrained  by  budget  or  deficiency  relationships. 
Formulation  5.2  provides  a  constrained  version  of  formula¬ 
tion  5.  1  as 


maximize  Sum/i  [  (FPVi)  (Xi)  ]  (5.2) 

subject  to 

Sum/i  [  (Aki)  (Xi)  ]  <  Bk 
0  <  Xi  <  1 
i  =  1 . . .  p 
k  =  1.. .  t 


where , 
FPVi 
Xi 
Bk 
Aki 

P 

t 


ith  final  program  value, 

proportion  of  ith  program  funded, 

available  budget  resource  in  time  period  k, 

full  funding  level  of  ith  program  in  time  period  k, 

number  of  programs,  and 

number  of  tine  periods. 


If  Bk  is  greater  than  or  egual  to  Aki,  summed  ever  all 
programs  for  each  time  period,  the  optimal  solution  to 
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formulation  5.2  is  also  at  Xi  e^ual  one  for  all  i.  This 
formulation,  however,  will  provide  a  tool  to  observe  the 
optimal  allocation  change  when  the  budget  level  Bk  is 
reduced,  thus  constraining  the  objective  function.  The  new 
optimal  program  priority  listing  will  not  necessarily  be  the 
same  as  the  ordering  provided  by  FPV.  After  the  linear 
program  computations,  the  vector  (FPV)  (X)  represents  the 
proportional  reduction  in  the  programs’  Final  Program  Value 
and  provides  the  revised  priority  listing  under  budget 
constraints . 

Formulation  5.2  is  illustrated  by  an  example.  Table  V 
lists  the  benefit  measurement  output  values  from  four  PDTSE 
programs  considered  during  a  partial  I1CDEC  process  test 
conducted  hy  the  Development  Center.  The  respective  budget 
allocation  data  from  the  RDT5E  POM  is  also  listed.  A 
constant  is  imbedded  into  the  linear  program  formulation 
which  defines  possible  budget  reductions  and  thus  a  tighting 
of  the  ccnstraints.  The  constant  is  added  by  replacing  the 
budget  ccnstraints  of  formulation  5.2  with 

Sum/i  [  (Aki)  (Xi)  ]  ♦  BR(Bk)  <  Bk  (5.3) 
wher  e , 

BE  =  budget  reducticn  proportion. 

Eight  progressivly  reduced  budget  conditions  are  used  in 
this  example. 

Table  VI  lists  the  resulting  optimal  solutions  and  rank¬ 
ings  for  each  of  the  eight  budget  conditions.  The  proportion 
cf  full  funding  for  each  project  changes  but  not  as  the 
MCDEC  prioritizing  vector  FPV  implies.  Although  the  FPV  of 
program  C0020  is  larger  than  the  FPV  for  both  programs  C0021 
and  CCQ82,  the  latter  two  programs  are  fully  funded  at  each 
budget  reduction.  The  values  of  FPV  alone  would  not  suggest 
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TABLE  V 

i 

1 

MCDEC  Portfolio  Selection 

Model  Example  Data  1 

J 

Program 

F?V 

Bud  get 

Data 

yr  1 

yr  2 

yr  3 

yr  4 

m 

C  1120 

7.32464 

47  47 

20874 

9176 

8987 

8588 

CO  020 

6.96  16 

21100 

22200 

30100 

33700 

7C50C 

CO  02  1 

4.79723 

4172 

4297 

4369 

4268 

3927 

C0C82 

4.21985 

361 

370 

330 

442 

443 

Budget 

Sum  =Bk= 

30380 

47741 

44025 

47397 

8385  9 

these  funding  reductions.  For  this  example,  observation  cf 
the  differences  in  the  programs’  budget  profiles  as  listed 
in  Table  V  would  imply  that  the  optimal  funding  in  a  budget 
constrained  problem  would  be  obtained  by  reducing  the  most 
expensive  program  first.  However,  in  a  larger  example  the 
optima]  solution  may  not  be  as  evident. 

Formulation  5.2  can  be  expanded  to  include  a  time  index 
for  the  funding  proportion  variable  X.  This  expansion  is 
formulated  as 


maximize  Sum/k  [Sum/i  (F PVi)  {Xik)  ] 


(5.4) 


subject  to 


where , 


Sum/i  [  (Aki)  (Xik)  ]  +  BR  (3k)  <  Bk 

0  <  Xik  <  1 

i  1 .  .  .  p 
k  =  1 .  . .  t 


BR  =  budget  reduction  proportion,  and 

Xik  =  proportion  of  ith  program  funded  in  tne  kth  period. 


The  data  values  for  FPVi,  Aik,  and  Bk  remain  the  same  as  in 
formulation  5.2.  Formulation  5.4  provides  greater  flexi¬ 
bility  in  funding  level  possibilities  and  provides  a 
sequence  of  program  rankings  for  each  subsequent  year.  Table 
VII  provides  a  summary  of  the  funding  proportion  solutions 
and  the  ranking  according  to  the  vector  (F?V)  (X)  for  each 
year  and  budget  condition.  The  most  expensive  program  C0020 
is  again  the  only  program  that  requires  less  than  full 
funding  except  in  the  most  extreme  condition  where  program 
C1172  is  also  less.  Observing  the  proportional  funding 
profile  of  program  CQ020  implies  that  the  first  time  period 
is  the  most  restrictive  in  each  budget  condition. 
Considering  funding  levels  in  each  time  period  allows  a 
greater  utilization  of  resources  available  in  the  less 
restrictive  periods. 


TABLE  VI 

Formulation  5.  2  Solution  Results 


1  .0 

.  95 

Proportion  of 
.95  .90 

Full  Budget 
.85  .80 

.75 

.50 

Vector  X 

C  1 120  XI 

1 . 0 

1.  0 

1.0 

1.0 

1.0 

1.0 

1.0 

.92 

COO  20  X 2 

1  .  0 

.  97S 

.89  2 

.  784 

.677 

.57 

.  462 

0.  0 

C0021  X3 

1 . 0 

1  .  0 

1.0 

1.0 

1.0 

1.0 

1.0 

1.0 

C0C82  X 4 

1 .0 

1.0 

1.0 

1.0 

1  .  0 

1.0 

1.0 

1.  0 

BE 

.00 

.  0  1 

.05 

.  10 

.  15 

.  20 

.25 

.50 

Ranking 

C  1 120 

1 

1 

1 

1 

1 

1 

1 

1 

CCC20 

2 

2 

2 

2 

3 

4 

4 

4 

C  0  0  2  1 

3 

3 

3 

3 

2 

2 

2 

2 

C  0082 

4 

4 

4 

4 

4 

3 

3 

3 

Notes 

1.  X  =  Proportion  of  program  funded  and  is  specific 
for  each  budget  reduction  case. 

2.  BR  =  Proportion  of  budget  reduction. 

3.  Ranking  is  according  to  the  vector  (FPV)  (X). 


The  Final  Program  Value  (FPV)  provided  by  the  MCDEC 
process  as  a  benefit  measurement  is  postulated  to  represent 
the  relative  program  importance  in  accomplishing  mission 
deficiencies.  The  primary  purpose  of  defense  systems  acqui¬ 
sition  and  Marine  Corps  RDTSE  as  presented  previously  in 
Chapter  II,  is  the  accomplishment  or  alleviation  of 
mission  deficiencies.  Concentration  on  a  portfolio  selec¬ 
tion  model  which  considers  only  the  FPV  and  budget 
constraints  may  net  produce  solutions  representing  the 
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greatest  accomplishment  of  individual  mission  deficiencies. 
The  next  two  proposed  formulations  consider  mission  defi¬ 
ciency  dcccmplis hment  in  terms  of  other  M  CD  EC  output  values 
under  separate  constraining  conditions.  A  third  additional 
formulation  concentrates  directly  on  the  number  of  deficien¬ 
cies  alleviated. 

Formulation  5.5  optimizes  at  a  level  in  the  ?CEEC 
benefit  measurement  model  where  each  deficiency  is  consid¬ 
ered  individually.  The  FP 7  is  constructed  from  the  previ¬ 
ously  presented  values  of  SEV  and  DDEV.  The  D3  EV  directly 
considers  each  deficiency  where  the  SEV  is  the  subjective 
evaluation  of  the  RDTSE  programs  only.  Equation  4.2  defines 
the  vector  DDEV  as  the  product  of  the  Deficiency  Relative 
Importance  Weight  (EFIW)  and  the  Program  Proportion  of 
Deficiency  Accomplishment  (PPDA)  summed  over  all  deficien¬ 
cies.  Letting  the  vector  PPDA  be  linearly  reduced  by  the 
funding  proportion  of  the  respective  program  provides  a 
means  to  optimize  the  mission  deficiency  accomplishment 
directly.  The  linear  program  to  maximize  the  F.DT5E  program’s 
DDEV  is  formulated  as 


maximize  Sum/i  [  ( DEE  Vi)  (Xi)  * 


(5.5) 


or  eg uivalently, 

maximize  Sum/i  [  Sum/j  [  (ERIWj)  (PPDAji)  (Xi)  ]] 
subject  to 

Sum/i  [  (Aki)  (Xi)  ]  +  ER(Bk)  <  3k 
0  <  Xi  <  1 
i  =  1 . ..  p 
j  =  1 . .. d 
k  =  1 . .. t 


where  , 


d  =  number  of  deficiencies, 

6  1 


DDEVi  =  ith  program's  deficiency  derived  ef fe ctiveness, 
DRIWj  =  jth  deficiency  relative  importance  weight,  and 
PPDAji  =  ith  program's  accomplishment  of  jth  deficiency. 

Additional  constraints  concerning  individual  mission  defi¬ 
ciencies  which  establish  lower  bounds  on  the  deficiency's 
accomplishment  level  may  be  included  in  formulation  5.5. 
These  lower  bounds,  however,  introduce  the  possibility  of  no 
feasible  linear  programming  solutions  existing  because  the 
lower  bounds  are  set  too  high.  Adding  a  new  variable,  Yj, 
which  represents  the  deviation  from  accomplishing  the  lower 
bound  alleviates  this  possibility.  With  the  variable  Yj 
included  the  lower  bcund  becomes  a  deficiency  accomplishment 
goal.  Including  the  additional  deficiency  accomplishment 
constraints  the  formulation  becomes 


maximize  Sum/i  [ Sum/j  [  (DF.IWj)  (PPDAji)  (Xi)  ]]  (5.6) 

subject  tc 

Sum/i  [  (Aki)  (Xi)  ]  ♦  EE  (Bk)  <  Bk 
Sum/i  [  (PPDAji)  (Xi)  j  *  (Yj)  (DA j)  >  DA j 
0  <  Xi  <  1 
0  <  Yi  <  1 

i  =  1 . .. p 
j  =  1 . ..  d 
k  =  1 .  ..  t 


where , 

D  A  j  =  jth  deficiency's  accompli  snmen  t  level,  and 
Yj  =  deviation  from  DAj  achieved. 


The  solution  to  formulation 
selection,  prioritization,  and 
information.  Program  selection 
according  tc  the  funding  level. 


5.6  will  provide  program 
deficiency  accomplishment 
criteria  can  be  defined 
If  the  ith  P.DT  &  E  program's 
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funding  level,  Xi ,  is  zero  it  follows  that  the  ith  EDTSE 
program  should  not  be  selected  under  the  constraints  estab¬ 
lished  in  the  linear  program.  RDTDE  programs  that  have  low 
proportional  funding  levels,  Xi,  serve  to  identify  FD15- 
programs  that  may  need  to  be  reevaluated  according  tc  their 
economical  accomplishment  of  mission  deficiencies  prior  to 
continued  investment. 

To  prioritize  the  RDTSE  programs,  two  possibilities 
follow  from  the  present  MCDEC  process  and  formulation  5.6. 

The  possible  prioritization  vectors  are 

[  (EEEV)  (X)  ]  (5.7) 

or, 

(X)  [  (DELK)  (DDEV)  ♦  (SEIW)  (S  EV)  ]  =  (X)  (FPV)  (5.3) 

where , 

SEEK  =  subjective  evaluation  list  weight, 

SEV  =  subjective  evaluation  value, 

DDIW  =  deficiency-derived  list  weight, 

DDEV  =  deficiency-derived  efficiency  value, 

FPV  =  final  program  value,  and 
X  =  program  proportion  funded. 

Equations  5.7  and  5.6  represent  vectors  that  could  be  used 
depending  cn  the  importance  given  to  the  vector  SEV.  The 
listings  weights  represented  by  SSL*  and  DDL'*'  are  assigned 
by  the  Prioritization  Working  Group  (PHWG)  to  compute  the 
FPV  in  phase  III  of  the  MCEEC  process.  If  full  funding  is 
available,  ranking  programs  by  equation  5. 8  is  equivalent  to 
the  current  dCDEC  prioritization  process. 

Formulation  5.6  also  provides  a  proportion  of  defi¬ 
ciency  accomplished  according  to  the  value 


Sum/i  [  (PPDA  ji)  (Xi)  *. 


(5.9) 


This  information  would  te  valuable  to  a  decision  maker  for 
identifying  deficiencies  that  are  not  fully  acccirp lished. 
The  additional  variable,  Yj,  provides  similar  identification 
information  concerning  deficiencies  not  achieving  their 
minimum  accomplishment  levels  as  set  by  the  constant  EAj. 

Tc  further  illustrate  the  linear  programming  model 
formulations  presented  in  this  section  a  larger  set  of 
Development  Center-prcduced  MCEEC  process  test  data  is  used. 
The  data  involve  158  mission  deficiencies  and  74  EDIEE 
programs.  The  proxy  budget  data  is  obtained  by  using  the 
same  year  RET&E, Navy  ICM. 

Table  VIII  shows  the  results  of  this  larger  data  set 
when  subjected  to  formulation  5.2.  The  table  lists  six 
columns.  The  first  two  columns  from  the  left  name  and 
describe  the  budget  conditions  used.  The  next  three  list 
the  number  of  RDT&E  programs  fully,  partially,  or  not  funded 
respectively.  The  last  column  provides  the  number  of  changes 
to  the  original  program  prioritization  list  when  using 
formulation  5.2.  The  results  depict  the  RDT6S  program  selec¬ 
tion  produced  by  the  linear  programming  model  solution  for 
maximizing  the  Final  Program  Value  (FPV)  subject  to  five 
progressively  restrictive  budget  cases.  As  should  be 
expected,  fewer  programs  are  funded  fully  as  budget  reduc¬ 
tions  are  imposed.  However,  if  the  FPV  prioritization 
listing  is  used  to  select  the  funded  programs,  the  optimal 
selection  obtained  through  the  linear  program  may  not  be 
produced.  The  right  hand  column  of  Table  VIII  provides  the 
number  of  programs  whcs  priority  position  is  lowered  from 
the  constrained  case.  This  column  contains  the  number  of 
programs  which  lost  their  priority  position  because 
(FPVi)(Xi)  <  (FPVb) (Xb) ,  where  i  represents  the  program 
under  consideration  and  b  represents  the  program  initially 


ranked  iomediatly  belcw  the  ith  program.  These  numbers  are 
an  indication  of  the  the  differences  in  selecting  EDTSZ 
programs  by  a  linear  programming  solution  rather  than  the 
initial  prioritized  list  produced  in  tne  dCDEC  process. 


TABLE  VIII 

Besults  of  Formulation  5.2  Kith  MCDEC  Data 


Priority  Changes  in 
F? v  Banking 


Case 

EE 

Program  F 
Full  Part 

u  nding 
None 

A 

0.  CO 

74 

0 

0 

E 

0.10 

72 

1 

1 

C 

0.  15 

71 

1 

2 

E 

0.25 

69 

2 

3 

E 

0.50 

61 

2 

11 

The  results  of  formulation  5.5  with  the  set  of  KC EEC 
process  cata  is  showr  in  Table  IX.  These  results  depict  the 
linear  program  solution  for  maximizing  the  program’s 
Deficiency-Derived  Efficiency  Value  ( DDEV)  when  subjected  to 
progressively  restricted  budget  constraints.  The  ncticable 
solution  difference  between  maximizing  DDEV  instead  of  FPV 
is  the  larger  quantity  of  5DT5E  programs  dropped  during  the 
DDEV  budget  reduction  conditions.  The  differences  between 
the  two  formulation  results  is  due  to  the  lack  of  the 
Subjective  Evaluaticn  Value  (SEV)  in  the  DDEV  solution. 
Assuming  that  the  DDEV  is  a  more  direct  and  objective 
measure  cf  FDTSE  program  accomplishment  of  mission  deficien¬ 
cies  tben  in  an  actual  application  the  partially  funded  and 
non-funded  programs  would  be  identified  for  further 


For mulation  5.6  further  restricted  the  maxi  mizat icr.  of 
Sum/i  [  (DDEVi)  (Xi)  ]  by  including  constraints  for  each 
mission  deficiency  in  addition  to  the  budget  constraints. 
Table  X  provides  formulation  5.6  results  when  using  the  same 
MCDEC  data.  In  this  illustration  the  jth  Deficiency 
Accomplishment  (DAj)  value  is  set  equal  to  a  single  guar.tity 
for  all  deficiencies.  This  value  for  DA  represents  a  defi¬ 
ciency  accomplishment  goal  which  the  model  user  would  like 
the  portfolio  of  selected  RDTSE  programs  to  achieve.  As 
previously  stated,  this  goal  may  not  be  feasible  with 
currently  considered  programs.  The  variable  Yj  in  formula¬ 
tion  5.6  provides  assurance  that  a  feasible  solution  will  be 
obtained,  and  it  provides  identification  of  the  deficiencies 
that  fail  to  meet  the  user's  goal.  If  Yj  is  equal  to  zero 
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then  the  jth  deficiency  goal/  EAj,  has  been  met.  If  the  Yj 
value  is  one  then  no  accomplishment  of  the  jth  deficiency  is 
provided  under  the  specified  set  of  constraints.  The  results 
cf  four  cases  are  shewn  in  Table  X.  Four  values  of  DA  are 
used  and  each  DA  case  is  farther  subjected  to  five  budget 
reduction  conditions.  For  each  case  shown  in  Table  X  the 
budget  reductions  impose  the  same  3DISE  program  selection 
portfolio  as  in  formulation  5.5.  The  two  priority  change 
columns  provide  the  number  cf  programs  that  lower  their 
priority  standing  as  a  result  of  the  linear  programming 
model  reducing  the  funding  proportion  Xi  to  obtain  an 
optimal  solution. 

The  deficiency  accomplishment  columns,  of  Table  X,  show 
the  number  of  deficiencies  that  obtain  the  goal  of  EA  in 
each  case  and  the  numbers  of  deficiencies  partially  and  not 
accomplished.  In  the  fully  funded  cases  all  HDTSE  programs 
are  selected  as  in  the  current  MCDEC  process  and  all  defi¬ 
ciencies  are  at  least  partially  accomplished.  Case  I-A  shows 
that  74  deficiencies  are  greater  than  or  equal  to  100 
percent  accomplished  and  84  deficiencies  are  less  than  100 
percent  alleviated.  Case  IV-D  shows  that  at  a  25  percent 
budget  reduction  87  deficiencies  are  accomplished  at  greater 
than  cr  equal  to  85  percent  while  63  deficiencies  are  less 
than  65  percent  alleviated  and  eight  deficiencies  are  not 
accomplished.  For  this  data  set,  these  results  imply  that 
even  when  all  programs  are  fully  funded  many  deficiencies 
are  net  accomplished  at  the  set  goal  while  others  are  usung 
resources  in  excess  cf  that  required  to  meet  the  goal.  For 
the  data  used  in  this  illustration  25  deficiencies  were 
accomplished  at  200  percent  or  greater  and  the  largest 
ever accc  trp  lishment  was  535  percent.  Numbers  representing 
deficiencies  move  ficm  the  DA  column  to  partial  and  not 
accomplished  columns  as  budget  reductions  were  imposed  that 
reduce  funding  fer  programs  that  support  the  specific 
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T ABIE  X 

results  of  Formulation  5.6  with  MCDEC  Data 


Case 

Program 

Funding 

Priority 
Chan  ges 

Deficiency 

Accomplishment 

E  A  EE 

Full 

Part 

None 

DDE  V 

FPV 

DA  , 

?art 

None 

Case  I 

A  1 . C  .  CO 

74 

0 

0 

0 

0 

74 

64 

C 

3  .  10 

71 

1 

2 

3 

3 

74 

33 

1 

C  .  15 

€9 

1 

4 

4 

5 

73 

84 

1 

D  .25 

60 

1 

1  3 

13 

14 

70 

83 

8 

Z  .  50 

50 

2 

22 

23 

24 

6  1 

87 

1  0 

Case  21 

A  .55.00 

74 

0 

0 

0 

0 

77 

81 

0 

3  .  10 

71 

1 

2 

3 

3 

77 

80 

1 

C  .  15 

69 

1 

4 

4 

5 

76 

82 

1 

D  .  25 

60 

1 

1  3 

13 

14 

73 

77 

8 

3  . 50 

50 

2 

22 

23 

24 

64 

84 

1  0 

Case  III 

A  .  SC  . 00 

74 

0 

0 

0 

0 

39 

69 

0 

B  .  10 

71 

1 

2 

3 

3 

38 

69 

1 

C  .  15 

69 

1 

4 

4 

5 

37 

70 

1 

D  .25 

60 

1 

1  3 

13 

14 

33 

67 

8 

E  .  50 

50 

2 

22 

23 

24 

76 

72 

10 

Case  IV 

A  .85  . CO 

74 

0 

0 

0 

0 

93 

65 

c 

3  .  10 

71 

1 

z 

3 

3 

92 

65 

1 

C  .  15 

69 

1 

4 

4 

5 

91 

66 

1 

D  .  25 

60 

1 

1  3 

13 

14 

87 

63 

£ 

E  .  50 

50 

2 

22 

23 

24 

30 

63 

1C 

deficiency.  In  all  DA  goal  cases,  one  mission  deficiency  was 
not  accc up  1  ished  at  any  percent  when  a  budget  reduction  of 
-in  cr  .15  was  imposed.  When  the  more  extreme  budget  reduc¬ 
tions  were  imposed  acre  programs  were  not  funded  and  more 
deficiencies  were  not  accomplished.  The  linear  programming 
solution  procedure  concentrates  resources  into  programs  that 
accomplish  deficiencies  with  the  highest  Deficiency  relative 
Importance  Weight  (DEIW)  and  does  not  prevent  over  accom¬ 
plishment  in  excess  cf  100  percent.  This  allows  an  optimal 
formulation  to  occur  with  a  portfolio  that  does  not  select 
EDT5E  programs  that  only  accomplish  deficiencies  with  a  low 
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ERIW.  As  shown  in  case  I-E,  61  deficiencies  were  accom¬ 
plished  at  cr  greater  than  1UQ  percent  while  10  deficiencies 
were  net  accomplished.  At  some  point  nonaccomplishment  of 
the  less  important  deficiencies  will  outweigh  the  benefit 
from  cveraccomplishment  of  the  more  important  def icienc ies. 

The  firal  linear  programing  formulation  proposed  here 
does  net  duplicate  the  MCDEC  process  in  any  obvious  way  but 
concentrates  on  maximizing  the  quantity  of  all  deficiencies 
accomplished.  The  following  formulation  uses  the  previously 
defined  variable  Yj  as  the  deviation  from  the  set  deficiency 
accomplishment  goal.  The  formulation  for  a  goal  of  100 
percent  fellows  as 


minimize  Sum/j  [Yj3  (5.10) 

subject  to 

Sum/i  [  ( Ak  i)  (Xi)  ]  +  EH  (Ek)  <  3k 
Sum/i  [  (PPEAji)  (Xi)  j  ♦  Yj  >  1.00 
0  <  Xi  <  1 
0  <  Yi  <  1 

i  =  1 . . .  p 
j  =  1 . . .  d 
k  =  1 . ..  t. 

Formulation  5.10  optimizes  deficiency  accomplishment  by 
minimizing  the  deviation  from  full  deficiency  alleviation. 
Programs  are  restricted  hy  budget  constraints  and  art 
selected  according  tc  the  proportion  of  deficiency  accom¬ 
plishment  they  impart  to  deficiencies  still  below  the  set 
goal.  Table  XI  summarizes  the  results  of  Formulation  5.10 
with  the  HCEEC  process  data  set.  In  general,  fewer  programs 
are  required  to  provide  the  same  mission  deficiency  accom¬ 
plishment  than  when  all  programs  are  fully  funded.  Met  until 
an  extreme  budget  reduction  is  imposed  does  the  guar.tity  cf 
1  :  ^ --cent  or  greater  accomplished  deficiencies  d<.c_  .. 


T  AEIE 

XI 

Be 

suits  cf 

Formulation 

5.  10 

with 

HCBEC 

Data 

C  a 

se 

?r 

ocram 

"'i.iori  ty 

- 

ici e re  v 

DA  > 

1. 

0C  Funding 

C  nan 

ges 

A 

plishirer.  t 

BF 

Fu  x-l 

Part  None  2 

DEV 

FPV 

>1.10 

Part  None 

A 

.  CO 

6  1 

5  3 

1  3 

13 

7-» 

3  4  0 

B 

.  10 

6  C 

£  3 

14 

14 

74 

84  0 

C 

.  15 

58 

7  9 

16 

1  5 

74 

34  0 

E 

.  25 

53 

£  15 

21 

20 

74 

34  0 

E 

.  50 

3  1 

10  43 

5  1 

51 

70 

83  0 

Ey  concentrating  on  the  number  of  deficiencies  accom¬ 
plished,  formulation  5.10  identifies  programs  which  may  not 
he  reguired  even  when  funding  is  available.  The  solution 
values  of  Y j  provide  an  identification  of  the  deficiencies 
that  require  more  3ETES  program  dedication  to  achieve  full 
accomplishment.  Table  XI  case  A  shows  that  for  the  same 
deficiency  accomplishment  results  as  case  I-A  of  Table  IX 
only  £1  programs  require  full  funding  and  five  need  only 
partial  funding.  Case  I-A  of  Table  IX  allowed  7u  fully 
funded  programs.  This  reduction  in  selected  programs  repre¬ 
sents  an  16.7  percent  total  funding  reduction  over  the  five 
budgeted  years  and  a  9.9  percent  funding  reduction  ir.  the 
aost  restrictive  year.  The  PDT&E  programs  not  funded  ir. 
Table  XT  represent  those  which  should  be  reevaluated  as  to 
their  extent  of  deficiency  accompli shaen t  prior  tc  selec¬ 
tion.  If  these  programs  are  not  selected  this  may  lead  to 
tne  recovery  of  resources  tc  provide  greater  accomplishment 
cf  the  partially  alleviated  deficiencies. 

Unless  the  values  of  DDEV  and  FPV  are  used,  formulation 
5.10  does  not  provide  a  means  to  compietly  prioritize  a 
listing  cf  F.DTSE  programs.  Formulation  5.10  does  separate 
the  programs  into  the  three  funding  classif ications  of 


70 


fully,  partially,  and  not  funded.  If  a  p  r  ion  t  in  at  i  cr.  i.; 
Enquire  I,  fully  furred  programs  wo  u:  logically  form  n>- 
highest  priority  and  the  not  funded  fro_,:ar:s  would  forr  th>:- 
iowest  priority  classification  set.  The  partially  funnel 
programs  would  fora  the  middle  priority  set  of  rrograms  ar.d: 
could  he  further  rarked  within  this  set  according  to  tn  dr 
respective  funding  prcporticn  Xi. 

C.  C I SCISSION 

Two  general  approachs  to  the  C .  _ .  .  ri  oritizatac:. 

process  using  linear  program,  u  .  ;  have  b  ..  . ropcsed. 

first  ar£roach  demonstrated  -.«.  IOl  Iiu  lali  ^  '  «  1  #  S  •  Z  /  0  •  / 

5.5,  and  5.6  maximized  the  MCDZC  process  values  concerning 
SDTSE  program  relationships  to  mission  deficiency  accom¬ 
plishment.  Each  of  these  formulations  have  an  unconstrained 
case  where  they  duplicate  the  currently  proposed  XCIEC 
process.  Formulation  5.10  represents  a  second  method  to 
approach  the  MCDEC  process  througn  concentration  on  the 
quantity  of  deficiencies  accomplished  and  selecting  programs 
as  reeded  to  obtain  an  accomplishment  goal.  These  approaches 
ar.d  formulations  illustrate  the  feasibility  and  f lex ir ility 
of  the  linear  programming  pcrtxolio  selection  meld  as 
applied  to  the  MCDEC  process.  The  results  listed  in  lades  I 
and  XI  show  that  for  small  tudget  reductions  both  a£proacnes 
select  programs  that  -rovide  similar  quantities  of  !dicien- 
c_es  accomplished  but  formulation  5.10  provides  *  dss 
c.stly  alternative  to  achieve  these  deficiencies. 

~c  accept  the  results  of  any  linear  programming  ■  >uch 
a  ;..agcr  assumption  must  be  accepted.  Linear  progr  idling 
assumes  that  the  relationships  of  coefficients  and  variables 
used  in  the  formulation  are  linear  or,  at  least,  that  linear 
functions  satisfactorily  approximate  the  actual  relation¬ 
ships.  Intuitively,  and  as  presented  in  some  of  the 


literature,  these  interrelationships  between  FDTZ  E  programs, 
resource  expenditures,  ar.d  benefits  nay  .not  be  linear  but 
instead  m ay  be  some  ether  functional  form.  The  linear  func¬ 
tions,  however,  do  provide  an  approximation  to  the  actual 
situation  which  is  u rderstandable.  The  proposed  f crmulaticus 
are  possibilities  which  can  serve  as  urogram  selection  tools 
and  net  as  decision  makers  in  and  of  themselves. 

Tc  further  investigate  the  linearity  assumptions,  the 
relationships  of  the  several  individual  coefficients  and  the 
two  prcpcrtional  variables  should  be  discussed.  The  vari¬ 
ables  Xi  and  Xj  which  represent  proportional  values  can  both 
be  readily  accepted  if  their  respective  coefficierts  are 
proper ticnally  divisible.  In  the  proposed  formulations,  tne 
budget  constraints  used  the  quantity  [  (Aki)  (Xi)  ]  tc  define 
the  budget  allocation  to  the  ith  program  in  the  kth  time 
period.  He  linear  assumption  maintains  that  a  program 
funded  at  a  proportional  ameunt  less  than  one  will  yield  the 
sane  prcpcrtional  benefit.  In  actual  applications  the 
linearity  assumption  may  not  be  appropriate  throughout  the 
entire  range  of  Xi  from  zero  to  one.  Some  programs  may 
require  a  proportional  funding  lower  limit  greater  than  zero 
to  remain  a  viable  fCTSZ  program.  For  this  condition  the 
ith  program  which  requires  a  minimum  funding  level  could 
have  the  respective  funding  proportion  variable  Xi  defined 
at  zero  and  over  a  rarge  from  the  minimum  funding  proportion 
to  one.  Another  condition  may  require  either  full  cr  no 
funding.  This  condition  could  be  achieved  by  defining  the 
variable  Xi  at  the  integer  values  of  zero  and  one  cr.ly  and 
utilizing  integer  programming  computations  to  find  the 
optimal  program  selection. 

A  similar  discussion  as  presented  for  the  budget  coeffi¬ 
cient,  Aki,  can  lead  to  acceptance  of  the  linear  assumptions 
concerning  the  coefficients  FFVi,  DDEVi,  and  FPDAji  as  they 
are  proportionally  reduced  by  the  variable  Xi.  First,  these 


coefficients  must  be  accepted  themselves  as  values  capable 
cf  retaining  their  benefit  measurement  properties  during 
linear  programming  computations.  Stevens  [Ref.  26]  proposed 
the  four  measurement  scales  of  nominal,  ordinal,  interval, 
and  ratio  which  are  helpful  to  discuss  the  properties  of 
these  coefficients. 

Briefly,  nominal  scale  measurements  represent  numerical 
identification  f cr  an  item  cr  event  such  as  an  autc  license 
plate  number  and  does  not  have  any  mathematical  properties 
associated  with  it.  Ordinal  scale  values  measure  rank 
ordering  only  such  as  the  priority  numbers  of  1  through  74 
given  tc  the  F.CTSB  programs  as  a  result  of  the  MC1EC 
process.  Interval  and  ratio  scales  maintain  a  comparative 
worth  between  values.  The  interval  scale  has  an  established 
interval  unit  and  each  specific  value  is  measured  by  the 
number  of  these  units  that  separate  it  from  other  measure¬ 
ment  values  above  or  below  it  cn  the  scale.  A  natural  origin 
is  net  always  established  in  an  interval  scale  even  if  a 
zero  value  exists.  The  zero  values  on  interval  scales  are 
assigned  to  arbitrary  points  such  as  on  the  Celsius  or 
Fahrenheit  temperature  scales.  Ratio  scales  have  a  natural 
origin  in  addition  tc  the  properties  of  the  interval  scales. 
Ratio  scales  are  capable  of  proportional  comparisons  cf  the 
numerical  values  such  as  stating  a  program  which  costs  two 
million  dollars  is  twice  as  expensive  as  a  one  million 
dollar  program.  To  maintain  the  same  measurement  properties, 
ratio  scales  can  withstand  multiplicative  transformations 
only.  linear  transformations  can  result  in  other  ratio 
scales,  however,  the  new  values  will  be  measures  cf  a 
different  attribute  than  the  first. 

In  linear  ecuatiers,  values  with  at  least  interval  scale 
properties  are  necessary.  In  linear  programming,  however, 
the  coefficients  require  the  additional  properties  of  ratio 
scale  measurements  tc  withstand  the  ratio  comparisons  of 
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coefficients  and  right  hand  side  values.  In  Chapter  IV, 
FPDAji  is  defined  as  a  straightforward  estimate  of  the  ith 
program's  proportion  of  the  jth  deficiency  accomplishment. 
If  these  estimates  are  accurate,  the  coefficient  PPDAji  can 
he  acceptable  as  a  measurement  on  a  ratio  scale.  The  DDEVi 
coefficient  is  a  summation  of  individual  proportional  trans¬ 
formations  of  each  FFEAji  based  on  the  Deficiency  relative 
Importance  Weight  for  the  jth  deficiency  (DEIWj) .  The  DDEVi 
can  be  accepted  as  a  different  ratio  scale  measurement  by 
accepting  as  valid  the  multi-attribute  utility  analysis 
which  is  used  to  derive  the  DRIBj.  The  FPVi  is  a  linear 
function  of  DDEVi  and  a  Delphi  process-based  Subjective 
Evaluaticn  Value  (SEVi) .  DDEVi  and  SEVi  are  both  adjusted  by 
the  confidence  factors  of  the  Deficiency  Derived  List  Weight 
(DDL  W )  and  the  ith  program's  Subjective  Evaluaticn  List 
Weight  (SEIWi) ,  respectively.  Although  not  specifically 
stated  by  Creed,  [Ref.  25],  all  DDEVi  in  the  sample  KCIEC 
data  set  were  adjusted  by  a  common  DDLW  while  each  SEVi  was 
adjusted  by  its  specific  SELWi.  Accepting  FPVi  as  a 
different  ratio  measurement  of  RDTSE  program  benefit 
requires  accepting  (SEVi)  (SELWi)  added  to  (DDEVi)  (DLLW)  . 
Although  beta  FPVi  and  DDEVi  might  be  accepted  as  ratio 
scale  measurements  of  F.DTSE  program  benefit,  because  of  the 
functional  change  between  them,  they  do  not  measure  the  same 
attributes  cf  the  ith  program.  The  vector  (SEV)  (SEllv)  also 
does  not  represent  the  same  qualities  as  the  vector  SEV 
because  cf  the  evident  change  of  SELWi  according  tc  each 
program. 

The  KCDEC  prioritization  process  proposed  only  the  FPVi 
to  represent  the  firal  relative  worth  of  FDTSE  programs  in 
the  accomplishment  of  mission  deficiencies.  If  tais  value  is 
acceptable  then  formulations  5.2  and  5.4  provide  examples  of 
linear  programming  portfolio  selection  models  which  will 
maintain  an  optimal  program  selection  portfolio  under  budeet 


resource  reductions.  If  the  subjective  evaluation  addition 
included  in  the  ?PVi  definition  is  not  acceptable  then  the 
multi-attribute  analysis  of  individual  deficiency  importance 
can  be  included  by  maximizing  Sum/i  [  (DDE Vi)  (Xi)  ]  as  in 
formulations  5.5  and  5.6.  These  formulations  allow  program 
selection  solutions  under  budget  reductions  which  may  net  be 
obtained  through  priority  listings  only.  The  last  formula¬ 
tion  approach,  which  does  not  directly  represent  the 
currently  proposed  !dCEEC  process,  appears  to  be  a  favorable 
EDI E-E  pregram  selection  model  as  shown  from  the  example  data 
set.  Formulation  5.10  provides  identification  of  programs 
requiring  ree valuation,  deficiencies  requiring  dedicated 
programs,  and  a  selection  of  programs  that  accomplish  the 
maximum  deficiency  alleviation.  It  should  be  noted  that  in 
all  the  proposed  linear  program  formulations  the  solutions 
are  optimal  computations  of  the  benefit  and  coefficient  data 
provided  and  can  only  be  given  the  confidence  inherent  in 
the  data  themselves. 
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VI.  CCNCIDSIONS 

A.  F I  SUITS 

This  thesis  has  investigated  the  MCDEC  Prioritization 
Process.  This  process  was  produced  by  the  Development 
Center,  MCDEC,  to  establish  the  relative  importance  of  EDTSE 
programs  prior  to  Marine  Corps  input  to  the  P.DTSE,N  PCM.  An 
investigation  of  the  literature  showed  that  the  MCDEC 
process  can  be  classified  with  similar  benefit  measurement 
models  currently  accepted  by  industry.  The  literature  also 
identified  ether  proposed  and  accepted  EDTSE  program  selec¬ 
tion  models,  classified  as  portfolio  selection  models,  which 
showed  possible  advantages  that  could  enhance  the  proposed 
MCDEC  process.  Several  MCDEC  process  linear  programming 
model  modifications  were  presented  and  showed  the  portfolio 
selection  model  advantages  of  maximizing  the  benefit  func¬ 
tion  when  subjected  to  resource  constraints. 

The  feasibility  cf  representing  the  MCDEC  process  as  a 
linear  programming  portfolio  selection  model  vas  illus¬ 
trated.  These  illustrations  raised  the  possibility  that  the 
Final  Program  Value  assigned  in  the  process  might  not  be  the 
test  measurement  of  EDTSE  program  mission  deficiency  accom¬ 
plishment.  Another  linear  programming  model  approach  to 
selecting  the  EDTSE  programs,  represented  by  the  example 
data,  was  also  illustrated  and  provided  an  economical  alter¬ 
native  for  maximizing  deficiency  accomplishment. 

Currently  a  mathematical  analysis  support  model  is  used 
as  a  tccl  in  the  Procurement  Marine  Corps  PCM  process  tut 
not  in  the  development  of  the  Marine  Corps  portion  of  the 


EDTSE, Navy  POM.  According  to  the  literature,  industry 
accepts  the  use  of  subjective  input  with  analytic  tools  for 


EDTSE  program  selection  applications.  The  MCDEC  process  as 
it  stands  or  with  the  portfolio  selection  enhancements  has  a 
tase  to  assist  as  an  analytic  tool  for  the  Marine  Corps 
responsibilities  in  the  RDT SE,  N  PCM  process. 

E.  F0F1EER  INVESTIGATIONS 

Several  areas  of  further  study  and  research  are  evident 
from  this  investigation.  The  variety  of  program  selection 
models  proposed  in  the  literature  each  have  advantages  -and 
disadvantages  which  reguire  further  investigation  as  to 
their  possible  application  to  the  Marine  Corps  P.DT5E  program 
selection  process.  The  data  gathering  and  compiling  tech- 
rigues  of  these  additional  models  reguire  study  as  well  as  a 
complete  analysis  of  the  data  and  process  results  from  a 
complete  trial  HCDEC  process  run.  The  most  difficult  further 
research  will  be  to  estimate  the  model  costs  and  perform  a 
cost  benefit  analysis  of  implementing  a  RDTS3  program  selec¬ 
tion  model  process. 

The  literature  review  chapter  presented  model  evaluation 
criteria  ard  limitations  suggested  for  industrial  PS  d 
program  selection  models.  Further  research  is  reguired 
into  the  applications  of  these  or  similar  criteria  and  limi¬ 
tations  in  the  context  of  Marine  Corps  applications  specifi¬ 
cally.  The  extent  cf  industrial  model  applications  tc  the 
Marine  Corps  will  be  limited  because  of  the  non-profit 
orientation  of  defense.  However,  the  other  portfolio  selec¬ 
tion  models  reviewed  in  the  literature  may  provide  valuable 
alternatives  to  the  presented  MCDEC  linear  programming 
model . 

Subjective  evaluation  of  RDTSE  programs  was  seer, 
accepted  as  a  necessity  in  the  literature  and  was  seer, 
proposed  for  use  in  the  MC  DEC  process.  The  mani pulaticn  of 
the  subjective  data  by  the  MCDEC  process  was  discussed  here 
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in  context  cf  measurement  scale  properties.  Further  investi¬ 
gation  into  the  subjective  values  and  their  true  measure¬ 
ments  is  needed.  Saaty  [Ref.  24]  and  Lindsay  [Ref.  27]  have 
proposed  different  methods  of  combining  subjective  judge¬ 
ments  into  quantities  that  maintain  measurement  scale  prop¬ 
erties.  Further  investigation  into  these  or  otner  subjective 
judgement  methods  may  provide  more  acceptable  alternatives 
than  the  Delphi  and  value  manipulation  methods  currently 
proposed . 

Formulation  5.4  illustrated  the  possible  resource  allo¬ 
cation  advantages  from  assigning  a  proportional  program 
funding  variable  to  each  year  in  the  planning  horizon.  The 
benefit  received  from  the  programs,  however,  may  alsc  change 
over  time.  Further  study  is  necessary  to  investigate  the 
program  benefit  measurement  values  used  in  the  portfolio 
selection  models  and  how  these  values  change  over  time. 
Additional  variables  and  benefit  coefficients  necessary  to 
describe  each  time  period  will  increase  the  computer  storage 
and  computational  requirements  and  this  increase  will  also 
need  investigation. 

The  costs  and  benefits  cf  implementing  tn e  proposed 
process  alsc  requires  further  study.  The  current  process 
software  was  developed  for  a  Tectronix  4054  minicomputer 
and  is  net  compatible  on  other  systems.  Resources  in  terms 
cf  manhours  and  money  will  be  required  to  upgrade  or  repro¬ 
gram  process  software  to  ensure  compatibility  with  mircccm- 
puter  systems  currently  at  HQMC  and  the  Development  Center. 
The  presented  linear  programming  computations  utilized  a 
Fetrcr,  Inc.  MFSIII  Dataform  package  as  run  on  an  IEil  3033 
computer.  Further  investigation  into  microcompu ter  linear 
programing  software  is  necessary  and  represents  additional 
costs.  If  large  linear  programing  packages  are  required  to 
enhance  the  process  additional  mainframe  usage  will  add 
costs.  Gathering  the  subjective  data  from  operational  and 
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ton-operational  units  during  the  MCD2C  process  survey  anal¬ 
ysis  represents  manpower  an3  TAD  costs  not  currently 
required.  The  total  monetary  benefits  are  possibly  the 
hardest  to  classify  cr  estimate.  The  value  of  valid  informa¬ 
tion  concerning  new  defense  systems  and  programs  as  they 
affect  the  mission  accomplishment  of  the  Marine  Corps  will 
need  further  research  to  .uantify. 


APPENDIX  a 

MCDEC  PRIORITIZATION  PROCESS 


This  appendix  displays  the  organizational  structure  for 
the  Development  Center,  Deputy  Chief  of  Staff  for 
Developmental  Coordination/  Prioritization  Wonting  Group, 
and  Chief  cf  Staff’s  Prioritization  Committee  in  Figures 
A.1,  A. 2,  A. 3,  and  A. 4  respectively.  The  remainder  cf  the 
appendix  details  the  phases  and  tasks  of  the  1CD2C  prioriti¬ 
zation  process  and  lists  the  specific  Development  Center 
agency  responsible  fcr  completing  eacn  step.  The  purposes 
for  each  fclloving  tasks  are  cuoted  from  Creed  [Ref.  25'. 
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Oe*e)oi>»ent 
Center 


Figure  A.  1  Development  Center  Structure. 


82 


Dates:  1  March  to  30  April 

Agency:  Planning  and  Evaluation  Branch 

Purpose:  To  provide  the  basis  for  justification  of 

EEC  projects  via  specified  Marine  Corps  deficien¬ 
cies  resulting  from  Mission  Area  Analysis. 


E.  Phase  II:  Prioritize  deficiencies  and  regu iremen ts , 

lash  1.  Mission  Area  Analysis  Results  Eriefings 


Dates:  1  May  to  15  May 

Agency:  Planning  and  Evaluation  Branch 

Purpose:  Tc  provide  the  Prioritization  Working 

Group  (PRWG)  with  Mission  Area  Element  (s) 
def iciencie s-criente d  briefings  in  preparation  for 
the  PRWG  MAA  prioritizations  [Phase  II  Task  2). 


Task  2.  Survey  Acalysis  Results  Eriefings 


Dates:  1  May  tc  15  May 
Agency:  Analysis  Support  Branch 

Purpose:  Tc  provide  the  Prioritization  Working 

Group  (PEW G)  with  FMF/major  command  and  agency 
derived  input  obtained  during  Phase  I,  concerning 
deficiencies. 


Task  3.  Design  of  the 


Deficiency  decision  tree 


Dates:  15  May  tc  15  July 


Agency:  Prioritizaticn  Working  Group  (PRWG) 


Purpose:  To  analytically 

attribute  utility  analysis, 
ordering  af  all  Mission  Area 
ar.d  an  associated  relative 
each . 


derive, vis  multi- 
a  consolidated  rank- 
(element)  deficiencie 
importance  weight  o 


C.  Phase  III:  Prioritize  FDD  programs. 


Task  1.  Program  Project  briefings 
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Dates:  15  July  to  15  August 

Agency:  Division  DEC’S  (scheduled  by  Dperaticr.s 

E  r  a  n  c  a) 

Purpose:  To  provide  preparatory  information  tc  the 

Prioritization  Working  Sroup  and  Chief  of  Staff’s 
Committee  for  consideration  in  subsequent  prioriti¬ 
zation  deliberations. 


Task  2.  Survey  Analysis  Sesults  Briefing 


Dates:  15  July  to  15  August 
Agency:  Analysis  Support  Branch 

Purpose:  Tc  provide  preparatory  information 
derived  from  FMF/majcr  commands  or  agencies  tc  the 
Prioritization  Working  Group  and  Chief  of  Staff’s 
Committee  for  consideration  m  subsequent  prioriti¬ 
zation  deliberations. 


Task  3. 
Pro  'ects 


Subjective 


Prioritization 


of 


?r  cgra 


Dates:  15  August  to  1  October 
Agency:  Analysis  Support  Branch 

Purpose:  To  obtain  a  subjective  prioritization  of 

BSD  Projects  «ith  associated  weighted  values  of 
each  reflecting  relative  worth. 

Task  4.  Deficiency- Derived  Prioritization  of  PS 
Ere jec  ts 


Dates:  15  August  to  1  October 

Agency:  Prioritization  Working  Group  (PRWG) 

Purpose:  To  obtain  a  prioritization  of  PSD  program 

projects  ranked  and  weighted  according  to  that 
project’s  capability  to  overcome  Mission  Area 
Element  deficiencies. 


Task  5.  Final  Pricriti zation  of  RSD  projects 

Dates:  1  October  to  15  October 

Agency:  Prioritization  Working  Group  (PRWG) 
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Purpose:  To  obtain  a 
program  projects  from 
and  deficiency-derived 


final  prioritization  of  R6E 
the  subjective-derived  list 
list  combined. 


Task  6.  Director  Development  Center,  Decision  Brief  for 
Sction 


Dates:  15  October  to  31  October 
Agency:  As  Directed 

Purpose:  To  present  the  recommended  RSD  program 

project  priority  for  decision. 


D. 

EOM  . 


Phase  IV:  Complete  MCEEC  R  5 


Task  1.  Application  of  funding 


D  programs  list  in  K^MC 


profiles 


Dates:  1  November  to  31  December 
Agency:  Operations  Branch 

Purpose:  To  determine  recommended  funding  profile 
tc  reflect  the  desired  prioritization  of  RE 
Erogram  Projects. 
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